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Introduction 



This Support Document for How’s it going? Monitoring progress in non-accredited language, literacy and 
numeracy learning, provides additional detail to supplement the Final Report. 

There were five phases of this project: Phase 1 comprised further interrogation of Dymock’s 
2006 NCVER research data, a literature review, and identification of possible research partners; 
Phases 2 and 3, included reporting of interviews conducted at each of the partner sites and the 
collaborative development of a draft portfolio of monitoring tools. The testing and validation of 
those tools was the major activity of Phase 4, culminating in a workshop with the partners and 
researchers. Phase 5 entailed preparation of the Final Draft Report for feedback from the project 
partners and the NCVER internal and external reviewers, followed by submission of the Final 
Report. 

The purpose of the Support Document is to provide: 

rir a full version of the literature review, and the possible bases for identifying the wider benefits 
of learning which emerged from that review; 

rir a set of tables to summarise illustrative comments from coordinators, tutors and students 
about wider outcomes of adult language, literacy and numeracy learning, categorised against 
those tentative bases; 

A a table to capture the perceptions of coordinators, teachers & tutors about the extent to which 
it might be possible to develop indicators for those bases; 

A a table showing the responses of practitioners and students to examples of possible 
instruments for monitoring progress in non-accredited language, literacy and numeracy 
learning; 

rir profiles of the five partner organisations for the project; 
rir partners’ reports on the trial of the selected instruments; and 
rir copies of the original and revised interview schedules. 

In addition, a pdf attachment includes as appendices the nine examples of instruments for 
monitoring progress identified during the literture review and to which practitioners and students 
responded., and from which the six trial instuments were derived. 
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Literature Review 



Introduction 

For the International Adult Literacy Survey, introduced in 1 994 in 22 countries including 
Australia, and now conducted every few years, literacy is defined as ‘the ability to understand and 
employ printed information in daily activities, at home, at work and in the community — to 
achieve one’s goals, and to develop one’s knowledge and potential’ (Tuijnman, 2001). ‘To achieve 
one’s goals and to develop one’s knowledge and potential’ have a personal dimension to them 
which is not quite so easily ‘measured’ as are language, literacy and numeracy skills per se. In fact, 
the definition implies that language, literacy and numeracy provision involves more than the 
acquisition of skills. Similarly, the renowned literacy educator Paulo Freire (1995: 65) suggested 
that education should affirm men and women as ‘beings in the process of becoming — as 
unfinished, uncompleted beings in and with a likewise unfinished reality’. The intent of this 
research is to explore whether the process of what Freire called ‘becoming’, particularly in regard 
to the development of self-confidence, can be monitored in meaningful ways for adults in 
language, literacy and numeracy programs. 



Client groups, learner needs and outcomes 

From recent research in Australia, a picture is emerging of a diversity of client groups and of 
learner needs in Australian language, literacy and numeracy programs. For example, there are low 
skilled disadvantaged clients (Castleton and McDonald, 2002), diverse literacy and numeracy 
needs within communities (Beddie 2004), and the particular needs of indigenous communities 
(Krai and Falk 2004). Furthermore, Miralles and Golding (2007) advocated differentiating among 
refugees in order to better meet their individual needs and aspirations. In a recent study of non- 
accredited language, literacy and numeracy (LLN) provision in Australia, Dyrnock (2007) found a 
diversity of groups and agencies that offered LLN programs, which he classified into four 
categories: community providers, English as a second language assistance, disability service 
providers, and accredited training. 

For the purposes of the current project, it was decided to focus on just one of these groups: 
community providers, in order to make the project manageable and because arguably community 
non- accredited language, literacy and numeracy provision is where some of the greatest needs for 
‘soft’ outcomes are encountered. In the 2006 survey (Dymock, 2007) 64 of the 125 respondents 
were identified as community providers on the basis that they nominated their primary role as 
either ‘General Adult / Community Education’, ‘Specific adult literacy/ numeracy improvement’, 
or ‘Community Information/Referral’. However some of these organisations may also offer, for 
example, accredited training and/ or English as a second language support. In order to clarify 
further the nature of the client groups and their learning needs amongst the community providers 
in particular, the responses to selected items from the 2006 survey for those 64 providers were 
re-analysed for the current study. 
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From that analysis, in relation to the characteristics of the learners in community LLN programs, 
it seems that overall about two-thirds of them are women, and the biggest group is in the 30-49 
years age bracket, prime family rearing and income-earning years, which may influence learning 
goals and life trajectories. The next largest is the 50-59 age group, also an active earning period 
for many adults. However, the perception of the providers is that the biggest motivation (about 
30%) for learners in such programs is a general need to improve their language and literacy skills 
for daily use rather than any specific needs such as employment. But employment related needs 
are still strong, relevant to about a quarter of the learners in these community adult LLN 
programs. Nevertheless, the general and the person-oriented needs (such as social interaction and 
developing self-esteem) comprise about 60% of the perceived reasons for participation. For the 
purposes of the present study, the breadth of ages and diversity of needs suggest that any 
instruments used to assess and acknowledge outcomes will have to allow for a range of individual 
purposes and perceptions of ‘progress’, as well as for personal growth, self-confidence and social 
interaction outcomes. 

The conclusion from the data in relation to motivations is that learners may be more concerned 
with meeting immediate needs than with ‘pathways’ to other education or training or to 
employment. About two-thirds of providers thought the numbers going on to either training or 
to employment were between under 10% and up to 50% of learners. Nevertheless, amongst the 
64 community providers, only one thought that none of the learners went on to other training, and 
only two thought none went on to employment. 

Providers were asked to indicate ‘realistically’ to what extent they thought learners developed 
LLN skills and self-confidence respectively as a result of the tuition they received. The ratings 
from the 64 organisations were based on a validated five-point Likert scale, and ranged from ‘Not 
at all’ to ‘Very much’. 

Table 1 : Perceptions of learner outcomes from non-accredited community adult LLN learning 

(n=64) 



Extent of improvement 


LLN Skills 
No. % 


Self- 

Confidence 
No. % 


Not at all 




- 




- 


Very little 




- 




- 


Somewhat 


14 


21.9 


4 


6.3 


Quite a bit 


34 


53.1 


18 


28.1 


Very much 


14 


21.9 


42 


65.6 


Not known 


2 


3.1 




- 


Total 


64 


100.0 


64 


100.0 



No. = Number of organisations 



As shown in Table 1, all the community providers responding to the survey believed that there 
were positive outcomes for the learners in both LLN skills and self-confidence. None of them 
thought that learners had progressed ‘not at all’ or ‘very little’ in either category. In respect of 
perceived skills development, more than half the 64 respondents thought that their students 
improved ‘quite a bit’ as a result of the teaching or tutoring provided, with the rest fairly evenly 
divided between ‘somewhat’ and ‘very much’. On the self-confidence aspect, some two-thirds of 
providers thought their students improved ‘very much’ through participation in their courses, 
with almost 30 per cent perceived to have improved their confidence ‘quite a bit’. Noteworthy is 
the finding that providers perceived a greater level of development of self-confidence than of 
LLN skills. An important question, which is a major focus of this present study, is what criteria 
providers used to make their decisions about this growth in self-confidence. The concept of self- 
confidence and its relationship to learning are explored in the section that follows. 
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Learning and self-confidence 

The link between learning and self-confidence is perhaps the most central one to understand 
what motivates and directs lifelong learning, such as further developing individuals’ language and 
literacy skills and engaging in further education. Although governments and employers are 
interested in marshalling individuals’ lifelong learning efforts towards particular goals, there is 
litde evidence to suggest that unless there is coincidence between those goals and the individuals 
that these will become the focus of individuals’ learning efforts (Billett, Smith & Barker 2005). 
Therefore, it is important to understand what directs these individuals’ lifelong learning and view 
them in terms of: (i) individuals’ interests and intentionalities; (ii) what governments and 
employers want learnt; and the relationships between the two. 

In the 2004 study of adult literacy programs (Eldred et al 2004) conducted by the National Institute 
for Adult and Community Education (NIACE) much is made of participants’ heightened self-esteem 
and confidence as being a consistent and unanticipated outcome of their participation in these 
programs. It was noted in this study that even when there were limited gains being literacy 
competence, the participants had a heightened sense of self. Moreover, it was noted that this legacy 
extended beyond the program and the individuals “seem to take their gains in confidence with them” 
(Eldred et al 2004: 3). However, not emphasised within the report is the rich association between the 
disposition of self-confidence and learning. Certainly, there are important and profound links between 
individuals’ self confidence and their learning, which goes beyond their interest in learning and 
extends to how and what they learn. That is, how individuals constme and construct knowledge from 
what they experience is shaped by their dispositions (i.e. values and beliefs), which include those 
about themselves and how they view the world. 

Dispositions, such as those comprising self-confidence or sense of self have been viewed as 
individuals' tendencies to put their capabilities into action (Perkins et al 1993a; b). From this 
perspective, the potency of self-confidence resides in the difference between what individuals may be 
capable of doing and what tasks they actually undertake, which includes how they engage in 
constructing knowledge or learning. The suggestion here for LLN programs is that success measured 
in terms of heightened sense of self leads to the likelihood of individuals being more ambitious and 
expansive in applying their capacities. Such a view looks to personal motivation (Hoffman 1986) as a 
means to engage learners in realising their full capabilities, which indeed is a recommendation from 
the NIACE (2004) report. 

Yet, beyond improving the tendency to utilise capacities to their fullest, dispositions associated with 
self-confidence play a more fundamental role in learning. Swain (2006) claimed that the majority of 
those participating in a maths learning program, changed through their participation, with some 
identifying specific changes to their aspirations and self esteem as a result of achievements through 
this program. They also shape how individuals constme and construct what they experience and 
therefore think, act and learn. For example, some individuals hold implicit beliefs about knowledge, 
considering “levels of intelligence” to be fixed, while others consider their levels can be developed 
further (Dweck & Leggett 1988). Such beliefs shape approaches to, and attitudes about, directing and 
securing their learning (Piaget 1981). In this way, the findings of the NIACE project suggesting 
important and enduring positive outcomes of participation in LLN programs that go beyond those 
anticipated by sponsors who maybe primarily concerned with measurable gains in literacy 
competence. 

The processes of and consequences for this shaping of learning by individuals’ dispositions arising 
from participation in LLN programs is seen to be particularly important given that contemporary 
accounts emphasise human learning as being a product of individuals’ construction and agency (i.e. 
constructivist perspectives). That is, we actively access and negotiate with and learn from what we 
experience and shape our construal and construction of that experience, rather learning being mainly 
a product of external sources. Within constructivist accounts there are conflicting claims about the 



Dymock and Billett 



7 




extent of role for both the individuals’ and external contributions to the processes of construal and 
construction of knowledge (i.e. learning). However, most views grant a role for the learner in these 
processes. Moreover, evidence suggests that individuals’ engage in this process as directed by their 
needs and capacities, and is mediated by their self-confidence sense of self. For instance, a study of 
small-business operators, found that they exercised their agency in particular and productive ways 
when engaged in learning about how to manage the Goods and Service Tax (Billett, Ehrich & 
Hemon-Tinning 2003). As with learning about literacy (Turner and Watters 2001), it seems that these 
learners were selective in how they engaged and in learning about the GST and what they learnt. That 
is, they deployed and exercised their personal epistemologies (i.e. personal and agentic ways of 
knowing and doing) as directed by their particular interests, imperatives and values. Yet, there were 
differences in the ways which these small business operators went about and engaged with learning, 
with the one key variable being their confidence in approaching this new learning. 

Similarly, Posner (1982) acknowledged the salient role of dispositions such as self-confidence to 
learning. He stated that "the beliefs, knowledge and abilities that students bring with them into the 
learning setting are a product of accommodations to their environments and form frames of 
reference which students use to assimilate new experiences" (Posner 1982, p. 345). Here, he also 
refers to earlier or pre-mediate experiences, as Valsiner (2000) described them, shaping how we view, 
engage with and learn from what we encounter. More than providing capacities in the form of 
concepts and procedures, these earlier or premediate experiences also shape how individuals 
subjectively (i.e. from their own perspective) construe and constmct what they subsequently 
experience (Billett 2003). Indeed, the unique set of experiences that comprises individuals’ life 
histories or ontogenies, lead to particular ways of viewing and engaging with the world which 
overtime shapes their ontogenetic development, in ways that might be quite personal specific and 
subjective (Billett 2003, Valsiner 2000). Consider, for instance, the distinct life histories that migrants 
and refugees bring to their participation in the programs (Miralles and Golding, 2007). This 
ontogenetic development is a product of moment-by-moment learning or micro-genetic development 
(Rogoff 1990) that comprises the ongoing process of the negotiations between what individuals 
already knows and what they experience, mediated by both the agency and intentions of the 
individual and the power of what is being suggested to them by them experiences of the social world 
(Billett 2006). 

In this way, the microgenetic development occurs throughout and is both shaped by and contributes 
to individuals ontogenies. Importantly, it is the legacy of these earlier experiences, including their 
disposition of subjective bases, which shapes how we engage in that ongoing development. In this 
way, individuals’ sense of self or self-confidence, arising from earlier experiences, shapes how they 
engage with subsequent experiences. Hence, how individuals engage in thinking and acting, for what 
purposes and intentionality, with what degree of effort (i.e. intensity) is shaped by these subjective 
bases (Billett, Smith and Barker 2005), such as individuals’ sense of self in that learning, and the 
knowledge they have of what they are learning about. 

Moreover, beyond individual learning, these negotiated processes of ongoing development also 
shapes how individuals engage in activities and, subsequently, enacted those activities: the constant 
process of remaking what we do as humans (Billett 2003). Such is the significance of these disposition 
contribution that, Rohrkemper (1989) questioned their separation from other forms of knowledge 
(i.e. conceptual and procedural) as did Vygotsky (1987) who saw this separation as being a key 
weakness in psychological theory. Indeed, Torney-Purta (1992) integrated dispositions within 
schematic stmctures, and Hoffman (1986) proposed that dispositional attributes such as confidence 
or sense of self have a direct influence on cognitive structures and activities, holding that they are 
embedded in and underpin both knowledge ‘that’ (i.e. conceptual) and knowledge ‘how’(i.e. 
procedural). Further, Gmsec and Goodnow (1994) proposed that affect and cognition need to be 
seen as influencing each other in ways that are not likely to be unidirectional or simple. Rather, they 
are likely to be bidirectional (i.e. negotiated) and complex, in their contributions to the ongoing 
learning and development. Hence, through their exercise these dispositional attributes stand to 
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underpin conceptual and procedural development, making them as potentially important general 
learning outcomes from participation in LLN programs. 

Another and related important consideration here is that dispositionally, self-confidence will be 
person-dependent and its capacity, means and potential to shape learning needs to be recognised on 
those bases. For instance, how adult learners come to engage with literacy and numeracy programs 
will shaped on basis and enacted in ways that are person-dependent to some degree (Dymock 2007). 
Moreover, how they engage in those programs and the learning that arises from them may be based 
on motives and processes that are distinct from those intended by sponsors, those who tutor in them. 
Hence, finding ways to understand the outcomes of literacy programs that can encompass the kinds 
of outcomes individuals want, arising from their participation and their achievement of these warrant 
careful assessment. Importantly, outcomes to be used to gauge the effectiveness of LLN programs 
likely needs to accommodate this person-dependent quality (Dymock 2007). 

So subjective and personally-derived attributes, such as self-confidence, are held to be central to 
individuals’ learning and their engaging in, remaking and, potentially, transforming their activities, and 
as such stand as important learning outcomes. Given the active role for learners in the constmction of 
knowledge, the degree to which individuals engage in, or withdraw from, a particular task will 
influence what they constmct and how they engage with that practice. For example, individuals are 
unlikely to engage enthusiastically in acquiring knowledge that they do not value, with quite the 
opposite likely to occur (e.g. Hodges 1998). That is, they may learn to disagree with, rebuff, or contest 
the very knowledge that they are being pressed to learn. In this way, given the importance of 
addressing and elaborating these disposition of attributes of sense of self or confidence, the fit 
between the goals of the LLN programs and those of the individual stand as another measure of 
these programs success. 

Certainly, dispositions, such as self-confidence, influence whether individuals value a particular 
outcome enough to be willing to participate in the effortful activity required to secure the requisite 
knowledge. For example, Dweck and Elliot (1983) reported that school students, with a performance 
orientation, may determine if participation in a school room activity will result in their "looking 
smart", which is quite a different goal from determining what they will learn from an activity. Also, 
strategic procedural knowledge (e.g. Gott 1989) - knowing how and when to apply knowledge — is 
also shaped by dispositional qualities, that are personally dependent. Yet this notion of strategic or 
higher order procedures has often been associated with the efficacy of securing goals, rather than 
whether the learner thinks they are worth securing (i.e. Dweck & Elliot 1983; Goodnow 1990; Tobias 
1994) or whether individuals possess the personal confidence, interest or motivation to engage with 
what they are experiencing (Belenky et al. 1986). Indeed, although a controversial text in so far as it 
has been held to disempower and misrepresent women as learners, the Belenky et al book argued that 
because of the subordinate roles that women had been assigned, this generated within women limited 
and narrow conceptions of what they were able to achieve and learn. While acknowledging that 
women should not be betrayed as being subjugated by social norms and practices, the point made in 
this book has implications for adult learners, and perhaps particularly those participating in literacy 
learning. That is, because of previous experiences with limited literacy skills, these learners may well 
perceive themselves to be limited in the scope of what learning is possible for them. Therefore, 
identifying outcomes which changed perceptions of the participants about the strategic potential of 
their learning stands as an important goal for LLN programs. 

Therefore, in considering the role of dispositions such as self-confidence in adult literacy and 
numeracy learning, the current thinking is to go beyond accounts that see self-confidence as merely 
energising cognition and exercising tendencies. Instead, it is important to acknowledge the role that 
disposition of qualities such as self-confidence play in personally-derived subjectivities such as self- 
confidence, and how this shapes what they experience and come to know or learn. As these attributes 
seem to empower individual agency and develop and fortify individuals’ personal epistemologies, they 
are central to general goals for learning. 
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Learner identity, personal agency, etc and their relevance to 
the concept of 'progress' in LLN 

Learner subjectivity 

From what has been proposed earlier, much of the basis for the learning associated with literacy, 
its processes and best need to be understood from the perspective of the individual. That is, their 
subject position. This is quite consistent with what is emerging as a set of alternative ways of 
assessing learner progress through LLN programs. For instance, Miralles and Golding (2007) 
emphasise that refugees might have particular needs and aspirations. They also will likely engage 
in LLN programs with particular kinds of subject positions. While some of these measures have a 
strong social flavour to them (e.g. social capital (Ballatti, Black & Falk 2006), measurable literacy 
competence (Perkins 2005), the majority have strongly personal dimensions. These include: Self- 
esteem and Personal Competence (NIACE 2004, ACFEB 2006, Dymock 2007); Engagement 
with others (ACFEB 2006); Agency and productivity (Watters and Turner 2001); Life trajectories 
(Ward & Edwards 2002); and Personal growth (ACFEB 2006, Dymock 2007, Swain 2006). 
Closely aligned with the earlier discussion about dispositions, and consistent with these 
alternatives of the potential for learning in LLN programs to be assessed against a range of 
outcomes, subjectivity stands as a common defining factor. 

Personal subjectivity comprises the conscious and non-conscious conceptions, dispositions and 
procedures that constitute individuals’ cognitive experience (Valsiner and Van de Veer 2000): our 
ways of engaging with and making sense of what we experience through our lived experience. 
This subjectivity is shaped by and also shapes encounters with both institutional and brute facts 
(Searle 1995): the contributions to experiences provided by both the social and natural world. As 
noted earlier, as a key component of our ontogenies, this experience continually develops 
microgenetically across our lifespan and informs how we construe and construct what we 
encounter through our active, agentic and intentional engagement with the social experience: the 
social world and also natural world (i.e. brute facts). 

Individuals’ cognitive experience is both deployed in and variously shapes and, at times, directs 
our conscious thinking and acting and is itself also renewed, reinforced, refined and transformed 
through that deployment (Billett, Smith and Barker 2005) as discussed in the section on learning 
above. Hence, individuals’ subjectivities comprise a set of conceptions, procedures beliefs and 
values and dispositions that are, in part, non-conscious (yet quickly become conscious when 
something we experience doesn’t fit) and, in part, conscious. Therefore, individual subjectivities 
and the allied concepts of sense of self and identity are essential to assessing the progress in and 
outcomes of individuals’ participation in LLN programs. 

These subjectivities find expression in two forms. There is the individual’s sense of self which 
guides the degree and intentions of our conscious thinking and acting strategically in seeking 
ontological security — making sense of the world (Newton 1998). Like Piaget’s concept of 
equilibrium, individuals’ sense of self is exercised to secure personal coherence in encounters 
with the social and brute world and to overcome disequilibrium it encounters. Giddens (1991) 
suggested the problem for the self is in maintaining its security in a culture that threatens its 
stability and the reference points for that stability. For instance, participation in LLN programs 
provide experiences that are likely to lead to challenges to individuals’ equilibrium. Yet, as 
Fenwick (1998) proposed, while permitting a role for individuals, this view positions them as 
anxiety ridden and their agency restricted to reflexive relations with culture, rather than 
individuals who have selves that are agentic and capable of mediating their own ontological 
security. Certainly, from studies of workers’ participation in working life over time and through 
processes of change (Billett, Barker and Hernon-Tinning 2004, Billett and Pavlova 2005, Billett, 
Smith and Barker 2005) the evidence suggests that while constrained and shaped by situational 
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factors, social practices and cultural mores, individuals are able to exercise their agency in ways 
aligned with being and maintaining themselves, albeit negotiating their sense of self through these 
processes. 

More specifically, the NIACE (Eldred et al 2004) report suggested that even when modest 
improvements to literacy competency are realised, participants enjoyed benefits personal growth 
beyond those indicated by measures of literacy competence. Yet, conversely Searle, Billett and 
Behrens (2005) found that adult learners’ initial participation in higher education was often 
marked by reliance upon the judgements of their lecturers about the worth of their learning and 
development, in this environment which challenge their sense of self and presented challenges to 
maintain their personal equilibrium in the form of new content and relatively independent 
approaches to learning, many of the students looked to the judgements of their lecturers’ marks 
and comments to make judgements about their worth. So their sense of ontological security is 
not found in either the personal or social but in negotiations between the two. 

Associated also with subjectivities is the concept of identity that has both personal and societal 
connotations. Socially, there are forms of institutional, normative and discourse practices that are 
associated with individual’s identity. Occupations, for instance, provide examples of these, and 
are ordered and valued in particular ways. So, there are societal expectations about and 
identifiable factors associated with those who wish to identify as a car mechanic, medical doctor, 
nurse, hairdresser and so on, as indeed there are about broader social categories (e.g. masculinity). 
The other account of identity is that aligned with how individuals present themselves to (i.e. to 
identify with) the social world and with which social practices they wish to be associated. This is a 
product of how individuals present and negotiate their self to the social world, in terms of what 
they do and how they go about it. Analogously, Cronick (2002) aligned individuals’ agency with 
personal control, which recognises the salience of control in conceiving and securing a sense of 
ontological security. In this way, identity is seen as an outcome, a narrative construction that is a 
product of this process and changes representing clear statements of progress of one kind or 
another. 



Agency 

As proposed earlier, individuals’ ontogenetic development . . . arises through a personally agentic 
epistemological process that is shaped through ongoing interactions with the social world. In 
turn, this influences how individuals engage with new experiences. The construal of these 
experiences is likely to be in some ways unique to individuals, because of their distinct personal 
histories. Indeed, it is likely that the level of interest and agency individuals direct to their learning 
of literacy and numeracy will be shaped by personal imperatives to secure particular kinds of 
change: 

the individual’s ontogenetic development . . . arises through a personally agentic 
epistemological process that is shaped through ongoing interactions with the social world. 
In turn, this influences how individuals engage with new experiences. The construal of 
these experiences is likely to be in some ways unique to individuals, because of their 
distinct personal histories. 

This notion of personal agency directing and shaping individuals’ learning holds promise for the 
present study. This is because it offers a basis to identify the extent to which it is perceived to 
change during participation in an LLN program, even if any changes cannot be directly attributed 
to the program itself. The challenge is to find ways of capturing the extent of such changes. 
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Approaches to assessment in non-accredited learning 

Dymock’s study (2007) and the additional analysis above provide some tentative bases for 
exploring how changes in the individual might be assessed beyond the measurement of language, 
literacy and numeracy skills. First, the perceived benefits of program participation were diverse 
for these learners, including personal growth and moving on to further education and training or 
to employment. Secondly, the needs of students were quite person-dependent, particularly in 
meeting the combination of needs that many learners experience. Thirdly, Dyrnock concluded 
(p.33) that in non-accredited ALLN provision in Australia, the ‘development of reading, writing 
and numeracy skills goes hand-in-hand with development of self-esteem’. So beyond the 
development of literacy and numeracy skills, other important personal outcomes might be 
achieved. 

This link has also been recognized in the wider field of adult and community education (ACE). 
For example, the Adult Community and Further Education Board (2006: 3) in Victoria said that 
that non-accredited learning is not only a pathway into accredited courses, but also a means to 
‘building confidence, resilience and self worth, enabling learners to make connections with family 
and the wider community’. Consistent with this, Clemens, Hartley and Macrae (2003: 47) 
characterised ACE outcomes as individual development outcomes, community development 
outcomes and economic development outcomes, but also observed: 

ACE agencies know they make a difference. They ‘see’ evidence of change, even dramatic 
change, in individuals, in communities and, to a lesser extent, in local economies. But they 
will never measure this change because they can’t isolate or quantify their contribution to 
change in one individual life, let alone succeed in the more complex task of isolating or 
quantifying their contribution to social capital and economic capital. 

In language, literacy and numeracy, the concept of wider impact has been explored by Balatti, 
Black and Falk (2006), who concluded that individuals’ involvement in literacy and numeracy 
courses produced social capital, but noted that the precise benefits varied according to age, 
English proficiency and background. However, this finding supports the person-dependent 
nature of appraising the benefits of LLN programs, particularly given the evidence of the kinds of 
personally significant changes likely to occur during individual’s life histories. Balatti, Black and 
Falk (2006: 23) reported that ‘self-confidence, self-esteem, and self-efficacy almost always 
accompanied changes in social capital or changes in improved technical literacy skills’. 

This notion of ‘dramatic change’ in individuals, and that individuals’ participation in LLN 
contributes to social capital, is consistent with Waterhouse and Virgona’s (2005: 28-9) conclusion, 
from a study of people who had succeeded despite the apparent handicap of inadequate literacy, 
that ‘literacy issues are about identity as much as [about] skills’. 

The concept of ‘identity’ in relation to LLN is very relevant to the issue of what outcomes are 
achieved and how they might be ‘measured’, and has been taken up in the present project as a key 
factor. A study of adult numeracy students in England (Swain, 2006: 3) found that almost three- 
quarters felt they had changed as a person in some way through learning maths, and that some 
students altered their aspirations as their sense of achievement and level of self-esteem grew, and 
concluded that : 

Although human agency may be fragile, particularly for those with little power, the 
students in this study were still able to make decisions that had the potential, at least, to be 
able to re-direct their lives. 

Swain suggested that students themselves perceive and direct their efforts towards such changes. 
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There has been considerable recent interest in the UK in the assessment of non-accredited 
learning in post- 16 years educational provision because of British government policy 
requirements (Greenwood, Hayes, Turner & Vorhaus, 2001). For example, following a 
government decision to fund non-accredited basic skills and English for Speakers of Other 
Languages (ESOL) courses as part of a plan to reduce the number of adults with basic skills 
needs, Grief and Windsor (2002) explored how such learning outcomes might be recognized and 
validated. They suggested there was a case for a nationally published format for individual 
learners’ records, but that it would need to provide for a ‘broad range of achievements including 
“soft” and “unintended outcomes” ’. In another study comprising interviews with 70 students in 
non-accredited adult literacy and numeracy learning, Ward and Edwards (2002:39) found that 
many of them did not seek qualifications and/ or feared assessment because of prior negative 
experiences at school. Torrance and Coultas (2004: 25) inferred from McGivney’s study (1992) of 
50 adult education organisations that for some learners, involvement in non- assessed activities 
might be a prerequisite for developing sufficient confidence to be able to engage in formally 
assessed courses. This finding is consistent with Dymock’s (2007) conclusion that the strong 
continuing demand in Australia for non-accredited ALLN courses is because these adults do not 
need or are not yet able to cope with certificate level training. 

Watters and Turner (2001: 4) found that ‘assessment’ was not a term used by learners in non- 
accredited learning, and that they preferred informal formative feedback from their tutors. The 
learners they interviewed were keen to talk about what they were learning, their feelings about 
that learning and the difference it was making in their lives: 

The most important dimension of assessment for learners was the satisfaction of knowing 

their own progress; proving this to others was understandably secondary. 

In the same vein, Wickert and McGuirk (2005:24) concluded from their review of literature that 
there was no consensus about the significance for disengaged learners of mapping their learning 
to specific literacy and learning outcomes. However, Janssen (2001:53) observed there is potential 
for conflict between students’ desire for informal qualitative assessment and funding bodies’ 
expectation of formal, summative assessment or at least ‘some quantitative indication of the 
levels of student “achievement” in non-accredited learning’. This raises the questions of what is 
meant by ‘progress’ and can it be measured? 

In a study of assessment in the UK learning and skills sector, Torrance et al (2005:28) observed 
that: 



Progression is conceptualised as both a ‘horizontal’ aggregatory process and a ‘vertical’ 
development process. In turn, both of these categorisations can also be subdivided into 
what might be termed intrinsic ‘progress’ — with respect to learning (or towards a learning 
goal) — and more extrinsic ‘progression’, with respect to moving from one accomplished 
achievement or qualification to another. 

They also noted (p. 33) that progression can also be conceptualised in terms of acquisition of 
social capital (‘personal confidence, social engagement, new or increased personal networks, 
community development and vitality’) and suggested that this was particularly apparent in the 
adult and community education sector. Attributes that build social capital, such as confidence, 
self-esteem, and trust, are what Falk (1999) referred to as ‘identity resources’, which he 
distinguished from ‘knowledge resources’. The development of identity resources are among the 
outcomes that Torrance et al (2005:84) called the ‘wider benefits of learning’, although they 
suggested that recording increases in self-confidence or social capital would be ‘challenging’. 

Nevertheless, the concept of ‘wider learning outcomes’ is pervasive in the literature, particularly 
in language, literacy and numeracy learning. For example, a Tasmanian survey (Department of 
Education, 2005:12) identified eleven ‘other outcomes’, including pathways to further study and 
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employment, better skills in researching and finding information, lower crime rates, and a 
significant increase in self esteem and in opportunities to participate in the community. A UK 
study of adult learners’ lives (Barton et al, 2004:101) identified the ‘wider benefits of learning’ as 
empowerment in the classroom and in life, new skills acquisition, and a change in attitudes to 
learning which added quality to life. Ward and Edwards (2002) concluded from their research 
with literacy and numeracy learners in north-west England, which used the ‘learning journey’ as a 
metaphor, that: 

Perhaps the most profound change for most learners interviewed was a massive 
enhancement of their confidence and self-esteem. This increased confidence had a 
significant impact on their learning achievements, attitudes to learning, aims and 
aspirations, ability to do real life activities and their social activities with other people. 

However a similar study in the north east of England (Gregson et al, 2004:34) did not observe 
the same ‘massive enhancement’ but did note that learners gained substantially from being able to 
share their learning experiences ‘in supportive dialogue’. These same researchers concluded that 
adult literacy and numeracy programs ‘need ... to be extended to include the progressive 
development of learner confidence, the skills of critical inquiry, strategies for learning to learn and 
the integration of learning aims with personal, social and psychological realities and ‘organising 
circumstances’ (Hamilton 1998) of everyday life’. 

Watters and Turner (2001:4-5) said that students in non- accredited learning believed they knew 
when they were making progress, but that the extent to which they were able to identify 
achievement of particular outcomes varied according to such factors as self-confidence, 
experience of adult learning and the factors themselves. In literacy and numeracy courses, Ward 
and Edwards (2002:4) found that most learners described learning gains ‘in terms of what they 
could now do in their lives’, while a smaller number referred to the acquisition of technical skills. 
Foster, Howard and Reisenberger (1997:13) attempted to meet the needs of different groups in 
non- accredited learning by classifying learning outcomes into ‘Outcomes for 
providers/ stakeholders/ funders’ (operational, and service outputs, service impact) and 
‘Outcomes experienced by the learner’ (personal, social, economic). Realistically, the examples of 
learner outcomes provided are not all positive, e.g. ‘wasted money’, ‘put off “education” ’ . 

Watters and Turner (2001) obtained only positive responses, mainly by asking learners in non- 
accredited learning what they considered they had gained from the experience. The list of 
beneficial outcomes included enjoyment and satisfaction, gains in skills, knowledge and 
understanding, a basis for further learning, a sense of well-being, increased confidence, seeing 
oneself and being seen differently, and seeing the world differently. Although these related to 
non- accredited adult learning in general, and therefore spanned a wide range of learner 
characteristics and motivations, the last four in the list are particularly pertinent to the present 
study of ALLN outcomes. 

According to Watters and Turner (2001), a sense of well-being came about through the emotional, 
psychological and physical benefits of learning: ‘Learning makes you feel good’. Increased confidence 
was demonstrated by being able to speak up in class, feeling at ease with technology, learn that 
it’s okay to take risks, and not being afraid of change. This study also found that a significant 
number of learners spoke positively about how learning had changed their perceptions ofi themselves 
as learners and as creative people and a realisation that ‘you don’t have to be intelligent to come 
to learning’. These attitudes were also part of seeing the world dijfierentlj as the learners’ views of 
other and beliefs changed. The researchers concluded (p.59) that the range of anticipated and 
unexpected benefits identified reflected the ‘diversity and complexity of the learners’ purposes 
and the range of ways in which non-accredited learning enhances adults’ lives’. They also 
suggested that ‘incidentally or sometimes deliberately’, the learning also enriched the lives of their 
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families, friends and acquaintances, a finding that is congruent with the discussion above about 
acquisition of social capital (Torrance et al, 2005) and ‘identity resources’ (Falk, 1999). 

Looking specifically at literature on non-academic outcomes in adult literacy programs, Westell 
(2005) identified five main outcomes: self-confidence, independence, attitude change, relationship 
and community building, and learning to learn. Westell said that every study mentioned self- 
confidence as crucial to learning but said it was not clear how this self-confidence was developed 
or what aspects of a literacy program promoted it. Under self-confidence Westell included ‘self- 
determination, self-direction, self-esteem, agency, choice, control, independence and standing up 
for oneself. She referred to a study by Niks et al (2003) in which the authors listed five categories 
of agency, self-confidence, control, choice, awareness and reflection. Another study (Bingman, 
Ebert & Smith, 1999) used the Rosenberg Self-Esteem Scale to map growth in self-esteem of 
adult literacy learners in Tennessee over a five year period. 

In Westell’s review, independence was exemplified by learners going to the shops on their own, 
living on their own, acting on their own and making changes in their personal lives which 
reinforced their independence. Under attitude change, Westell (p.10) noted Fingeret’s (1994) 
conclusions that learners in two adult literacy programs not only improved their reading and 
writing abilities, but often described new literacy practices at work and shopping, along with 
‘culture-related impacts, including changes in the culture of families and gender-related changes in 
behaviour’. Westell observed that it was not clear whether changes in attitude were a specific 
result of the educational intervention or also attributable to participation in a diverse group of 
learners. 

Under relationship and community building, Westell (p.ll) quoted Manning’s (2003) finding from a 
review of outcomes of US adult literacy programs that ‘participation in literacy programs leads to 
a significant increase in participation in other community organizations’. Her evidence for learning 
how to learn as an outcome is less persuasive, but she quotes Beder’s (1999) finding from a national 
US study that ‘in general, adult literacy education has a positive influence on participants’ 
continued education’. 

In another discussion of ‘progression’ and the wider benefits of adult learning, Nashashibi (2004: 
ii-iii) distinguished between ‘progress’, achievement’ and ‘progression’, as well as ‘impact’: 

Progress in learning is an increase, improvement or deepening of knowledge, understanding 
or skills. 

Achievement is what learners do with what they learn; it is the outward and visible sign of 
progress. It may take the form of passing an examination but it is observable in the 
actions, products and/ or behaviour of learners wherever active learning takes place. ... In 
non-accredited learning, achievement of a learning gaol is achievement of the aim as 
expressed in agreed individual or group learning objectives. 

Progression is movement as a result of learning. It is purposeful and takes the learner into a 
new context or activity. It may, for example, be movement into further learning, 
employment, freelance work or new voluntary roles. 

Impact is an inclusive term covering the overall effect of learning provision on those it is 
designed to benefit — the learners and potential learners and the communities in which 
they live. 

There is some overlap in the meanings of these terms which may cause confusion in trying to use 
them for assessment purposes in non-accredited ALLN. More useful for those purposes may be 
Nashashibi’s (2004: 27) list of the wider benefits of learning: gaining confidence and enhancing 
self-esteem, maintaining or improving physical or mental health, ‘keeping fit for learning’ — able 
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to use it when life change requires it, developing local social involvements and more tolerant 
attitudes — contributing to the fabric of society, using new skills to benefit the family or 
community, and improving the environment. Nashashibi noted that such benefits can result from 
participation in a learning activity as much as from completion , and that ‘engagement in learning is 
not all future oriented’, a comment that is very relevant to ALLN learners in non-accredited 
community programs. She also deduced from Schuller et al’s findings (2004) that the impact of 
learning might be thought of as a continuum with ‘sustaining learning’ at one end and 
‘transforming learning’ at the other. Schuller et al (2004 in Nashashibi, 2004, 29) said: 

Education transforms people’s lives but also, less spectacularly, enables them to cope with 
the multifarious stresses of daily life as well as discontinuous and continuous social change 
and contributes to others’ well-being by maintaining community and collective life. 

This is a conclusion from Schuller et al’s (2004) research into the benefits of learning generally, 
and may not apply quite so fully to those in ALLN programs. However, their matrix classifying 
the effects of learning into ‘Personal change’, ‘Self-maintenance’, ‘Community activism’, and 
‘Social fabric’ may be helpful in devising a set of indicators of progress in ALLN learning. 

Another way of trying to map the wider benefits of learning has been in identifying ‘soft 
outcomes’. Dewson et al (2000:2) described soft outcomes as: 

. . .outcomes from training, support or guidance interventions, which unlike hard 
outcomes, such as qualifications and jobs, cannot be measured direcdy or tangibly. Soft 
outcomes may include achievements relating to: 

Interpersonal skills, for example: social skills and coping with authority 

Organisational skills, such as: personal organisation, and the ability to order and prioritise 

Analytical skills, such as: the ability to exercise judgement, managing time or problem 
solving, and 

Personal skills, for example: insight, motivation, confidence, reliability and health 
awareness. 

According to Dewson et al (2000:2), the term soft indicators can be used to refer to those 
achievements that indicate progress towards an outcome, but they observed that this may be a 
subjective judgement. Dewson et al (2000:2) also used the term distance travelled to refer to the 
progress that a person makes ‘towards employability or harder outcomes’ and suggested for 
example that ‘indicators (or measurements) of soft outcomes can be used as tools for measuring 
distance travelled towards labour market participation’. 

Butcher and Marsden (2004:6) criticised what they saw as the narrowness of this linking of soft 
outcomes with hard ones, because ‘such an approach narrows the interpretation of social 
inclusion to employability and educational achievement and ignores vast swathes of the 
community and voluntary sector working with clients who may never access jobs or educational 
qualifications’. They also noted (p. 6) a study (Gaffney and Humm, 2002) that reported concerns 
from community organisations that the development of ‘measurement systems’ would turn 
volunteers into ‘professionals’. 

Assessment tools in non-accredited learning 

In considering how soft outcomes and distance travelled might be ‘measured’, Lloyd and 
OSullivan (2003: 5) identified five common elements in the practices of nine different 
organisations: a set of target indicators (e.g exercising self-discipline or assuming responsibility 
for oneself at work), a scoring system (usually in the form of a scale from three points to ten 
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points), baseline and subsequent interviews to assess progress, a system for reporting results, and 
training staff for using the system. Anderson, Foster and McKibben (2006: 1) developed the Soft 
Outcomes Universal Learning (SOUL) Record in which soft outcomes are divided into three 
areas: ‘attitude’, ‘personal/interpersonal’, and ‘practical’. By measuring changes in outcomes in 
these areas over time, ‘organisations are able to show the distance learners have travelled’. The 
SOUL Record uses 21 statements (7 for each of the three areas), with a five point rating scale for 
each, where 1= strongly disagree, and 5=strongly agree. Included under ‘Attitude’ is ‘1 am a 
Confident person’, under ‘Personal/Interpersonal’ is ‘I enjoy working as part of a team’, and 
under ‘Practical’ is ‘1 carry out any tasks that I am set or set myself. Three different ‘smiley’ faces 
at the top of the form ask the learner to indicate ‘how do you feel today?’. 

Adding the value of the rating for each of the 21 statements provides an overall score and users 
are recommended to use the tool at the beginning, midpoint and end of an intervention in order 
to be able to compare scores. Anderson et al (2006) claimed the SOUL Record had been used 
successfully in mental health, with sex workers and homeless young people, and ‘can be used 
effectively with clients with low literacy levels’. 

The SOUL Record is another version of a variety of approaches that have been developed to try 
to meet the UK Government’s requirement for documenting non-accredited learning. Many of 
these have adopted ‘individual profiling’, and a range of real-life examples is presented in A 
practical guide to measuring soft outco?nes and distance travelled (Department for Work and Pensions, 
2004). There has also been some UK interest in the assessment in the related area of ‘soft skills’ 
(Simpson, 2006) but these appear to be more in relation to accredited learning and the attempts 
to date to be problematic. 

There have been numerous instruments developed over the years to assess the development of 
literacy and numeracy skills (Brooks, Heath & Pollard, 2005; Soifer et al, 1990: 170-177), 
including in Australia the National Reporting System (Perkins 2005). However if, as the research 
so strongly suggests, the development of self-confidence is a key aspect of adult learning, new 
ways need to be found to monitor learner progress in areas beyond language, literacy and 
numeracy skills. 

Eldred (2002: 6) attempted to capture the elements of progress and achievement in adult literacy 
learning through the concept of success-, ‘the challenge is whether success is indicated by 
measurements based on external standards or a less easily measurable, more personal issue of 
achievement of individual targets and increases in confidence’. 

In Eldred’s (2002: 9) study, adult literacy tutors at a UK further education college perceived 
success in terms of: 

Increases in confidence, motivation, assertiveness, and feeling better about themselves; 
Demonstrating learning through a review of goals or targets; 

Using something which has been learned; 

Receiving feedback from friends or family; 

Doing something the student could not do before 
Attending regularly 
Indicating through assessment or tests 
Aspiring to do something else 
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Taking responsibility for learning 
Finding out by asking the student 
Receiving computer feedback. 

While the items in these lists are described by Eldred as ‘indicators’, the question remains as to 
what criteria the tutors used to assess such aspects as ‘increases in confidence’, ‘using something 
which has been learned’, and ‘taking responsibility for learning’. 

The majority of students interviewed by Eldred for the same study strongly asserted they were 
successful in their learning, and provided examples of this success. She classified the students’ 
responses into literacy indicators and those which suggested differences in life (p.14). Literacy 
indicators included reading the newspaper, talking to people in shops and offices, reading and 
writing and spelling more or better, and reading road signs. Eldred said (p. 14) that the students 
‘appeared to compare their previous and current literacy behaviour, in order to assess their 
success’. Indicators of differences in the learners’ lives were mainly described (p.14) in terms of 
changes in behaviour, including reading and purchasing newspapers and magazines, writing 
letters and cards at home, and using reading and writing at work. It is clear from the two lists that 
there is considerable overlap in what Eldred has termed ‘literacy indicators’ and ‘differences in 
life’ indicators. More distinct in the latter were student mentions (p. 14) of ‘increased confidence, 
doing new things, experiencing greater independence, and comments from family or work 
colleagues’. 

Eldred (2002) found that that ‘success in literacy learning is not necessarily evidenced by 
achieving standards set by external bodies but by goals which tutors and students discuss and 
agree’ (p. 27) She said that a ‘significant number’ of students appeared unable to achieve the 
external standards set, even after several years study, but that almost all students reported 
increases in confidence. This is compatible with Dymock’s (2007) finding, discussed above, that 
providers in non-accredited community ALLN perceived greater gains in self-confidence than in 
language, literacy and numeracy skills. From her research, Eldred concluded: 

Growth in confidence is a key outcome of successful literacy education and it should be 
recognised, assessed, recorded and celebrated in the process of teaching and learning. 

There have been numerous attempts in the past 25 years or so to capture the wider benefits of 
ALLN learning. For example, Good and Holmes (1978, p.3) in their book, How’s it going? (from 
which the title of this study takes its name) advocated a three-level assessment of reading, writing 
and spelling: ‘Beginning’, ‘Not bad’ and ‘With ease’, an acknowledgement that assessing LLN 
progress is related to students’ perceptions of themselves as learners as well as to their skills 
development. At about the same time, Charnley and Jones (1978) proposed five overall criteria 
for measuring success in language, literacy and numeracy which extended the conception of skills 
development into how those skills were applied: affective personal achievements (e.g. ‘an increase 
in confidence associated with literacy skills’), cognitive achievements (e.g. improved reading and 
writing skills), enactive achievements (e.g. reading newspapers, using writing skills at work), socio- 
economic achievements (e.g. participation in civic duties), and affective social achievements (e.g. 
better relationships within and outside the tuition group). Using these criteria in a national study, 
Brennan, Clark and Dymock (1993) identified a diversity of learner oriented outcomes among 
Australian LLN students. They concluded (p. 67) that ‘the value of literacy skills for what they 
meant personally or socially or economically and what this could permit the person to do or to 
be, was incorporated into the learners’ expectations’. 

However, the dominance of competency-based training and associated regulatory frameworks in 
Australian education and training in the past decade and a half, appears to have diminished the 
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significance of these personally-oriented outcomes because they cannot be easily measured. 
According to the Australian Council for Adult Literacy (2004, 7), in LLN ‘Australia needs a 
reporting framework that reports on progress against a number of relevant social and personal 
indicators’. The principles underpinning the Victorian Adult Community Further Education 
Board’s (2006) A-Frame, ‘An ACE framework for non-accredited learning’, reflect this need for a 
broader approach. Although not intended as an assessment document, one of the questions asked 
of tutors is: ‘What significant personal development for learners was observed: in confidence, in 
leadership skills, in attitudes?’ (p. 1 5) . 

In a recent British study (Eldred et al, 2004) that explored the link between learning and 
confidence, the authors made the point (p.4) that: 

In non-accredited learning, the challenge is to clearly capture individual and group 
achievements, in order to demonstrate to learners, tutors and funders what has been 
gained. If gains in confidence are as significant as many learners and tutors appear to 
suggest, ways of evidencing them seem to be important. 

Dymock (2007, p.3 1) found that ‘ways of evidencing’ in non-accredited LLN in Australia varied 
considerably. From a national survey of 125 providers and seven case studies in three states, he 
concluded: 

Only around one-quarter of the providers used formal assessment tools .... Amongst the 
rest, a combination of small assessment tasks and perceptions of progress based on 
observations and student feedback, often verbal, seemed widespread. This informality 
means that there is not a very rigorous assessment of student learning in many programs, 
particularly those using volunteers one-to-one. On the other hand, the range of 
motivations identified in the survey and in the interviews suggests that a more rigorous 
approach to assessment may not be appropriate. And if, for example, the development of 
self-confidence is seen as a worthwhile outcome, how might that aspect be assessed? To 
what extent should student self-evaluation be considered a reliable means of assessment? 

In considering the implications of the findings, Dymock (2007, 36) went on to say: 

it is unfair to expect coordinators and volunteers to jump through too many accountability 
hoops, but there does seem a need for a review of assessment practices, particularly in 
those programs where assessment is a litde loose or relatively unmonitored. In fact 
‘assessment’ may be too strong a term — what is necessary in some programs is a more 
careful approach to monitoring of progress, and the range of tools available for such purposes 
should be reviewed by individual coordinators to ensure that the one/ s selected for their 
particular program is /are the most appropriate. 

In Australia, the ‘measurement’ of outcomes has for some time been a thorny issue in the LLN 
field because it often pits personal against institutional purposes. Initiatives such as the Adult 
Literacy and Numeracy Competency Scales (Griffin & Forwood 1991) gave way to more student 
centred-models, eventually resulting in the development of the National Reporting System (NRS). 
This widely used tool, mandatory in the government-funded WELL and LLNP programs, is an 
instrument for describing language, literacy and numeracy outcomes. It was not, however, 
designed for assessing the range and kinds of broader outcomes in non-accredited community 
programs of the kind identified in the literature (above). The application of the NRS also requires 
a level of background knowledge of language, literacy and numeracy sufficient to be able to 
appreciate its effective use (Perkins 2005). Therefore it may be impractical for the numerous 
volunteer tutors practising in the community learning sector. Even under a proposed new 
framework, the purpose of the NRS would continue to be about levels of language, literacy and 
numeracy development (Perkins 2005), thereby missing the important personal imperatives and 
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goals which appear to be so important for individuals engaged in and learning through these LLN 
programs. 

A similar approach to the NRS seems to have been adopted in the United States by the Equipped 
for the Future (EFF) Assessment Consortium (2004: 1), led by SRI International and the Center 
for Literacy Studies at the University of Tennessee, in developing assessments for standards on 
‘what adults need to know and be able to do to be equipped for daily life in the 21 st century’. 
Researchers identified 16 standards under four categories of skills: communication skills, 
decision-making skills, interpersonal skills, and lifelong learning skills. These assessments are 
intended to ‘allow teachers to measure and report on how well adults can use the integrated skills 
processes that make up the EFF standards to reach goals and satisfy purposes in their lives’ (p. 1), 
so appear to have broad intentions, but they are based on performance measures which in 
Australia, the UK and New Zealand would be seen as more closely aligned with competency 
standards, i.e. with accredited learning. 

One attempt to assess growth in levels of confidence through non-accredited learning was 
through another NIACE project in the UK: Catching confidence (Eldred et al, 2004). The authors 
reported (p. 1 5) that the terms ‘self-esteem’ and ‘confidence’ were often used interchangeably in 
the literature. However they saw confidence as relating to ‘ability to do things’ and believed that 
this ability depends on possessing the knowledge or skills to do something, and that confidence 
can vary from situation to situation. ‘Self-esteem’ they saw as relating to how people view 
themselves and their feelings of self-worth as well as their ability to act. To them confidence is an 
aspect of self-esteem. In the study, tutors suggested that self-esteem is hidden within a person 
and ‘difficult to evidence’, whereas confidence is easier to identify through changes in behaviour 
and attitude (p. 55). Eldred et al (2004: 57) concluded: 

Without increases in confidence, many adults will remain non-participants, not achieving their full 
potential in personal development as well as skills and qualifications. The importance of non- 
threatening first-step learning which gives learners time to gain confidence, [and] develop their 
identity as successful learners, and supports diverse aims and aspirations is vital. 

The research showed (p. 31) that increased confidence resulted in changes in self-esteem, body 
language, ability to speak out, heightened life aspirations, ability to learn and aims for progression, 
relationships with family and friends, community activity and activism, work ambitions, 
performance and relationships. 

The challenges of assessing the extent of changes in confidence was summed up by Turner and 
Watters (2001: 117): 

We can identify the promotion of self-confidence as goal, and go some way towards forming 
judgments about learners’ self-confidence at the end of a course. But there is nevertheless plenty 
of hard work to be done on specifying exactly what it is that we are assessing, and exactly how we 
are in a position to verify that a learner’s self-confidence is as she or her tutor says it is. 

Eldred et al (2004: 21-24) attacked the issue through the development of a ‘confidence grid’. The 
‘Catching Confidence Grid’ comprises a matrix with twelve statements down the left-hand 
column, intended to capture different aspects of confidence, including ‘1 am confident when 
meeting new people’, ‘1 am confident when writing things down’, and ‘1 am confident 1 can do 
the things I want’. Across the page are four columns, each with a venue/ situation as a heading: 

‘At a learning centre’, ‘At home’, ‘Socially/ with friends’, and ‘At work/ out and about’. The 
learners were provided with cards, each marked with a different symbol for ‘highly confident’, 
‘confident’, ‘not confident’, and ‘very low confidence’, and asked to think about where they 
would place these in response to each of the 12 questions., and to discuss these proposed 
responses with fellow students. When they were satisfied with their choices, stickers with symbols 
the same as on the cards were available so they could confirm their decisions on the grid. The 
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intention was that the exercise would take place close to the start of a course and at the end so 
that the extent of change could be seen visually from the placement of the stickers. 

While the authors concluded the grid activity was a ‘powerful tool for supporting learners to 
recognise changes in confidence in and beyond learning situations’ (p. 24), it is clear from some 
of the feedback reported from tutors that there was some ambivalence about the value of the 
grid, and whether all learners were capable of recognising the extent of the changes in confidence 
that had occurred (pp. 23-4). Some learners found the level of language of the statements 
difficult, and some appeared to need more time to reflect on their responses. 

Eldred at al (2004: 57) concluded that ‘work should be carried out to develop best practice in 
framing confidence growth as a learning outcome and methods of recognizing and recording in 
this area’. This current study is intended to contribute to that work. The researchers are conscious 
however of Grief and Windsor’s (2002: 63) warning: 

We need to be cautious that the desire to make systems robust and ensure the credibility of data 
on learner’s achievements does not encourage the development of systems that resemble that of 
award schemes and qualifications. By doing this we lose the very features of non-accredited 
learning for which this option is currently valued. 



Some bases for revised approaches for monitoring learner 
progress in LLN programs 

Drawing on the approaches reported in the literature surveyed above, there appears to be a 
number of possible bases for monitoring progress in non-accredited community language, literacy 
and numeracy learning in terms of growth of confidence and related attributes, including: 

Self-confidence and personal competence — degree by which self esteem and personal 
confidence have changed, and in what ways, as a result of participation in literacy program 
(ACFEB 2006, NIACE 2004, Dymock 2007, Ward & Edwards, 2002, Gregson et al, 2004, 
Butcher & Marsden 2004, Foster et al 1997, Watters & Turner 2001, Bingham et al 1999, 
Nashashibi 2004, Schuller et a 2004, Dewson et al 2000, Eldred 2002, Charnley & Jones 1978, 
Dutton et al). 

Engagement with others - degree by which engagement with others has changed, and in what 
ways, as a result of participation in literacy program ( ACFEB 2006, Ward & Edwards 2002, 
Butcher & Marsden 2004, Fingeret 1994, Schuller et a 2004, Dewson et al 2000, Charnley & 

Jones 1978) (e.g. family, community). 

Attitudes to learning — extent of change of attitude towards learning and future learning and 
ability to learn how to learn (Ward & Edwards 2002, Watters & Turner 2001, Beder 1999, 
Nashashibi 2004). 

Agency/ pro-activity — degree by which individuals’ agency and pro-activity have changed, and 
in what ways, as a result of participation in literacy program (Swain 2006, Watters & Turner 2001, 
Falk 1999, Niks et al 2003, Schuller et a 2004, Brennan et al, Charnley & Jones 1978). 

Life trajectories - degree by which individuals’ life trajectories have changed, and in what ways, 
as a result of participation in literacy program (Swain 2006, Miralles & Golding 2007, Waterhouse 
&Virgonia 2007, Ward & Edwards 2002). 

Personal growth/ personal change — extent to which a learner perceives they have grown as a 
person as the result of engagement in a LLN program (Eldred 2002, Swain 2006, Turner & 
Watters 2001, Schuller et al 2004). 
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Social Capital — extent of changes in community participation, involvement in networks 
(Ballatti, Black & Falk 2006, Foster et al 1997, Torrance et al 2005, Manning 2003, Nashashibi 
2004, Schuller et al 2004, Charnley & Jones 1978). 
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Possible bases for identifying wider 

benefits of learning 

Seven factors were identified from the literature review in Phase 1 as possible bases for indicating 
the wider benefits of learning: (i) self-confidence; (ii) engagement with others; (iii) attitudes to 
learning; (iv) agency/pro-activity; (v) life trajectories; (vi) personal growth; and (vii) social capital. 
In the Phase 2 interviews, the coordinators, teachers, tutors and students were specifically asked 
about what they perceived as wider learning outcomes, i.e. beyond the development of language, 
literacy and numeracy competence. Examples of the responses, mostly by coordinators and 
tutors, but also by learners (L), are provided below under each of the seven categories. 

1. Changes in self-confidence and personal competence 

The participants were asked about the ways in which changes in self-confidence and personal 
competence were evident through the learners’ participation in non-accredited LLN programs. 

In Table 2, the illustrative examples of outcomes are presented in the left hand column; in the 
right column, a statement or statements have been made to crisply capture those reported 
outcomes, a pattern that is repeated in tables 3 to 8. 

Table 2: Changes in self-confidence and personal competence 

Illustrative data Learning outcome 

I think coming here, my confidence actually has built up quite a bit. I’ve learnt Improving personal capacity 

things that I thought were actually impossible to do. I’ve actually achieved more in and competence 

my maths than I actually thought I could ever actually do, and the same with my 
literacy class as well. (L) 

My biggest problem before was worry about what other people would think and More engaged; improved self- 
hiding when I was writing [at work]. Now, I know what it says and if I need help belief 

they’ll explain it to you. I’ve got the confidence, that’s all I wanted. (L) 

Before I couldn’t do anything, my husband do everything. So I was even scared to 
go to shopping; I was afraid when the lady in the shop ask me something because 
very often I couldn’t understand, so now I don’t have this problem, I don’t feel 
shy and even if I don’t understand something I don’t feel, you know, shy and I 
don’t have this feeling. So it’s much, much better. Much, much better. (L) 

I’d ask him to do things and he would do that and more, ... so he pretty much More autonomous; better 

took the reins on very, very early, within about three or four lessons. I could see communication 

amazing improvements in his self-confidence and just his communication as well. 

He’d tell me that he always went to the same deli for lunch . . . because of his Opens up options and 

anxiety. But then he started going to different delis and different shops and things alternatives 
like that, which was all new for him. 

One of my students . . . mentioned being able to write birthday cards and write the Enhanced participation in 
gift tags on Christmas gifts. You know, she said that was . . . the first Christmas everyday tasks 

she’d been able to do it and she was thrilled. 



More autonomous and self- 
directed 
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You see that progression from the eye contact to the yes/no answers to the 
engaging in conversation to the feeling comfortable sitting in the reception area and 
interacting with the volunteer [receptionist] . 


Enhanced engagement with 
others and improved self- 
esteem 


My student had the confidence to go for an interview for a job and then get the 
job, but also . . . it’s a big step for her to go up to the railway station and hop on the 
train and go all the way to [city] to get another train to go over to [suburb] to work 


Enhanced participation in 
everyday activities and greater 
self-belief 


From the responses shown in Table 2, a number of recurring themes emerge. Most commonly 
there is evidence of greater engagement with others and participation in activities by these 
learners than had previously been the case. This outcome is closely aligned with a capacity to be 
more autonomous and self-directed in activities. Hence, there are consistent indications that 
changes in self-confidence and personal competence were evident in both greater participation 
with others and in social circumstances and in greater independence in terms of being competent 
and confident to undertake more autonomous action on the part of these learners. 


2. Engagement with others 




The participants were also asked whether there had been any changes in terms of their 
engagement with others as a result of that participation in the non-accredited literacy and 
numeracy program (Table 3). 


Table 3: Engagement with others 




Illustrative data 


Learning outcome 


Now I would say since I was coming to school I don’t have much worry if I have 
to go to ring the office of the doctor, anywhere; there is no more that hesitant to 
speak, to say something, but that’s what, as I said, my confidence grew in that part 
there, I’ve got no worry to go anywhere, to the doctor, to anywhere. (L) 


Empowering the individual to 
engage more fully in the 
community 


They won’t speak freely because, you know, they might fear having a conversation 
and they avoid morning tea, for example ... So you can tell, as their confidence 
grows they join in and, you know, that’s another indicator. 


Developing the capacity to 
engage comfortably in 
conversations with others 


She’s got her confidence to say what she feels but in a more sensitive way. She’s 
more aware of other people now because . . . she can afford to worry about other 
people’s feelings because she’s more relaxed and hasn’t got to guard herself. 


Developing the capacity to 
engage with others 


[Student] has cerebral palsy and he’s severely disabled but his mind still works quite 
well. . . . I’ve seen a real improvement in his confidence when he comes here. 
When I first started with him he was quite shy. As you can imagine he’s had some 
very difficult times over his life and ... he has been ridiculed and so he’s a bit self- 
conscious and he’s not sure of how people are going to take him, but he’s really 
come out of himself, I think, coming here; he talks to people. 


Improved sense of self through 
being more competent in 
communication 


I’ve noticed in the surveys of students the effect on the family because so many 
times they say “I can read bedtime stories to my kids now that I couldn’t do before. 
I can help kids with the homework.” So that’s . . . got two effects — it’s helping the 
kid and showing them that literacy in the family is a good thing . . . but it’s also such 
a big thing for the self-esteem of the parent who felt inadequate before. 


Improved capacity to engage 
more fully in home life 


The responses shown in Table 3 have a number of common themes. The first is a sense of 
empowerment which entitled engagement in a range of activities which the learners might not 
have otherwise participated. Aligned to this is the claim that, for some learners, they had become 
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more competent and had masted more appropriate forms of communication there by improving 
their capacity to engage with others. Thirdly, is a recurring theme of an improved sense of self 
which was a product of the program and permitted broader engagement with others and in 
activities which the learners previously had not participated. 

3. Attitudes to learning 

The participants were asked about whether there had been changes in their attitudes towards 
learning as a result of that participation in the non-accredited literacy and numeracy program 
(Table 4). 

Table 4: Attitudes to learning 



Illustrative data 


Learning outcome 


When I took this program over, I found students stayed in this program. They 
weren’t confident enough to branch out into further learning opportunities. . . . 
Now I’d say that students are confident in their own ability to venture out to 
TAFE, to university as well. 


Provided a basis to consider 
alternative and other training 
options 


Their language and literacy has improved but it also empowers them to make that 
next step because they feel confident to face a larger [educational] organisation, to 
cope with all the forms, all the bureaucracy. . . 


Empowering the individual to 
participate in a higher level of 
educational activity 


I see that out of ten people one person goes on to some sort of further education; I 
think that’s a brilliant outcome. 


Improved educational pathways 
for some 


[From Year 10] my plan is to ... get Year 12 and if I get good results from Year 12, 
good number, go to uni to do about politics and law. (L) 


Developing belief in capacities 
and strategies for supporting 
progress 


He actually said to me at the end of last year “I don’t think I need you anymore. I 
hope you think that’s a good thing.” And I said “I definitely think it’s a good 
thing.” He was going to enrol in a TAFE course as well ‘cause he was a welder by 
trade but I don’t think he’d actually finished his official apprenticeship because the 
theory freaked him out a bit, so he was going back to do that. 


Competence to re-engage with 
educational activities that were 
previously a cause of anxiety 
and failure 


At first I wanted to be a paramedic . . . but two friends are in it and now to be a 
paramedic you have to have a degree. So it’s just made it like almost impossible; 
and then I talked to my friends even more and I’m not sure if it’s the line I want to 
go . . . It’s alright coming to a road accident once in a while but not like full on. So 
I actually want to become a fitter now and take up an adult apprenticeship. (L) 


Capacity to consider options 
and identify which are most 
likely to be achievable 


A key theme emerging from the set of data in Table 4 is the broadening of the participants’ scope 
of what was possible for them to learn. They seem to be empowered to widen their horizons and 
engage in options that previously they would not have considered pursuing. Here also is the 
issue of confidence again., that is, the confidence arising from participation in the program was a 
basis for engaging more widely, and in one case engaging in an environment where the individual 
previously had not enjoyed success. 


4. Agency/pro-activity 




The participants were asked about changes in their sense of self empowerment and ability to 
engage with and influence people (Table 5). 
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Table 5: Agency and pro-activity 
Illustrative data 



Learning outcome 



I’ve finally realised that I can be my own person and if I don’t want to go with Improvement in sense of self 

them I don’t have to. ... I guess they know I’ve put my foot down and I’ve found a and confidence to be assertive 

part of who I am and not afraid to say it. (L) 

She was too frightened to go to Sydney to see her daughter, because she was 
frightened she wouldn’t be able to understand what it said on the board at the 
airport and she’d miss her plane, she wouldn’t even get in a taxi on her own, and it 
was just self-confidence. And we worked on all of those things for about six 
months ... I just saw her grow just in doing that one little step, . . . what that lady 
learnt in that six months was just outstanding and you couldn’t mark it on a 
certificate. 

I remember one woman telling me how humiliated she was initially when she 
fronted up at the front desk in a TAFE college wanting a language and literacy 
course, and there wasn’t anything sort of that fitted but she said that it was just 
faceless and nobody really understood her, so she’d retreated and wasn’t going to 
be able to get past that and so was going to stay in this limited field of activity. But 
when she came upon the [LLN] program and the tutor was able to improve her 
literacy. That helped her ... be able to speak out for herself. She then went back to 
the TAFE college and said “This is what I want” and got into something in the 
end. 

I think that agency and that proactivity is seen in those students who have children 
at school and that they can actually go up to the office and they can ask for what 
they want, they can write a note to school and say “Give that to your teacher”. . . . 

Some of the ones that are at the latter end are proactive in looking for what sorts of 
courses they might like to be involved in or what they would like to be able to learn 
about. 

Consistently, across the responses reported in Table 5 are references to enhancement of the 
participants’ sense of self and confidence to be more assertive in their personal and community 
lives. That is, through participation in these programs, these individuals have developed a greater 
sense of self and employment which was opened up the prospects for them to engage in activities 
with greater confidence and a feeling of them being more in control of that interaction. 

5. Life trajectories 

The participants were asked the degree by which the life trajectories including their goals and 
ambitions had changed as a result of their participation in the literacy and numeracy program 
(Table 6). 



Confidence to negotiate with 
institutions and privileged 
individuals on behalf of family 



Confidence to consider 
alternatives and assess the 
scope of potential activities and 
be assertive 



Enhance sense of self and 
confidence to engage in options 
and alternatives 
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Table 6: Life trajectories 



Illustrative data 


Learning outcome 


There’s a fair few things like that I wouldn’t mind becoming. I have a lot of friends 
and either family sometimes come up to me about either problems or just — 
problems in life in general and 1 mean I’ve thought about becoming either a 
counsellor or a social worker .(L) 


Confidence to consider and 
pursue new career trajectories; 
recognition of strengths 


I feel much more confident and much more independent and now I can plan my 
life, first of all I’ve got a full time job . . . and currently I’m enrolled in Certificate 
III in Business at TAFE . . . I’m doing my English classes through TAFE as well. 

(L) 


Confidence to realise personal 
goals through participating in 
educational processes 


She had goals at the beginning which were totally unattainable. They were personal 
goals, not changing the world goals. But I think she’s taking off . . . she’s learned to 
take bite sizes rather than the whole plate at one time, she’s learnt that the first 
step, there’s a first step and a second step and you can do things in sequence in 
smaller bites to achieve the big thing, you don’t have to do it all at once. 


Capacities to be realistic about 
career goals and have plans to 
progressively achieve these 
goals 


I found a lot of students change their goals as they go because they might be 
progressing faster than what they thought, so their confidence grows and they aim 
higher than what they started with, but because they come at the very beginning 
and it’s all new territory and they’re a bit intimidated by it all they start with, I think, 
very low, realistic goals and then after a couple of months they think ‘oh hey, I 
might be able to do this’. 


Competence to realise personal 
goals and develop new ones 


A young woman with an intellectual disability . . . was working at a [charity] shop 
and I think she was sorting clothes and hanging clothes on racks and helping 
people coming into the shop. But she wanted to use the cash register . . . and so she 
worked with her tutor and . . . her maths was great and she uses the cash register, 
she does the pricing now of the clothing, she reads to the other workers at 
lunchtime from the community newspaper and shares stories with them. And her 
goal is now to do the banking, . . . and the next is she wants to take over the 
management of the shop. (C) 


Competence to achieve 
personal and vocational goals, 
and develop new vision for self 


Students . . . have said that they now feel able to take on promotion, more 
responsible positions at work, ‘cause before they just haven’t. Or, when they’ve 
been offered promotions they’ve said “Oh, no, don’t want to — I don’t know how 
to do that. No.” But after increasing their literacy levels they’ve then said “Well 1 
took that promotion at work” or “My boss says I’ll be able to move up to this now 
that I can do that.” 


Competence to achieve 
workplace goals 


The consistent theme from these data is the increase in personal competence to pursue individual 
goals and trajectories. This included having the competence to actually understand what was 
realistic and achievable, and understand the need for attending to significant goals incrementally. 
All this suggests that the kinds of outcomes arising from participation in these programs were 
those informing the participants of ways which allow them to be more selective, strategic and 
focused in securing their personal goals. 


6. Personal growth/personal change 




The participants were asked about personal growth or personal change that had arisen through 
their participation in these literacy and numeracy programs (Table 7). 
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Table 7: Personal growth - personal change 



Illustrative data 


Learning outcome 


You see a whole outlook — they’re different, very different. Even their physical 
appearance can often be different by the time they leave. 


Personal growth 


I’m not a shy person but when I came in Australia I was very shy because I 
couldn’t communicate and now . . . because I can speak a litde bit more I’m 
recovering my self confidence (S) 


Enhance self-confidence to 
engage with others 


fin her own country] she knew the language, the culture, everything, and suddenly 
she’s transplanted here and it’s all different; and the language she thought she knew 
at school wasn’t the same when people were talking it, and all the usual cultural 
things, so she was a bit shattered. But then eventually she started coming good 
again and I kept encouraging her, telling her she was very intelligent and it was just 
a question of time and a bit of application and you can do anything you like, you 
know, and she’s starting to believe me, I think. 


Self-confidence to embrace a 
new country and new culture 


She’s picking up and she’s now starting to go off and do little bits by herself which 
is more growth and more self-confidence. 


Increased self-confidence and 
autonomy 


Their level of confidence, you can actually see that grow . . . As the barriers come 
down they see that . . . it’s friendly, it’s a relaxed pace. Yeah, they grow - before 
long you really see them flapping their own wings. 


Enhanced self-confidence and 
engaging with others 


To start with she was in a sort of a sense emotionally . . . numbed, because of the 
stress levels associated with all this stuff, and to have been released from that she’s 
beginning to feel she’s got time and energy to get in touch with her own feelings 
now. 


Developing the capacity to be 
herself and less anxious 


I’m . . . thinking of students in this program . . . who’ve said to me themselves, and 
their wives who’ve commented too, that they’re far less aggressive at home. ... So 
that would be a real sign of personal change there. 


Confidence which leads to the 
need to be less dominant with 
others 


Certainly the body language, the attitude, . . . you could see the head up . . . The self- 
motivation was huge. 


Self-confidence and improved 
body language 


A lot of the . . . students do comment on that, in that they’re saying how much 
better they feel and that they can cope with things. 


New sense of wellbeing and 
capacity to engage with others 
and the world they encounter 


We’ve had a few students [who] get to a point where they start to know what they 
don’t know, you know, they were happy in their ignorance before and then they 
realise what they don’t know and then that becomes quite confronting and there’s 
lots of decisions that they have to make and there’s a big responsibility from the 
organisation, not just the tutor on their own, to manage that. 


Confidence to question issues 
that concern and confound, 
and to take responsibility 



The most consistent response here is that participation in these programs had led to 
enhancements of the participants’ self confidence. This was repeated and reiterated in the 
participants’ responses. Nuanced instances here included the confidence not to be boldly 
assertive, because this was unnecessary, coping better with demanding circumstances and an 
increasing questioning of things which previously might have been accepted or taken for granted. 
In these ways, the participants were suggesting that personally important strategic outcomes arose 
through their participation in these programs. 
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7. Social capital 



The participants were also asked what changes had arisen for them in terms of how they 
participated in the community and their networks (Table 8). 



Table 8: Social capital 



Illustrative data 


Learning outcome 


Well the environment here is actually fantastic. Like everyone socialises with 
everyone, the people here are great, like you can go on your breaks and you can 
sort of have tea and coffee with one another and have conversations with one 
another. 


The capacity to engage with 
others socially in a positive way 


When she first arrived she was staying at home a lot and feeling a bit miserable and 
what the hell am I doing here, what have I done? And yeah, just as soon as she 
started getting out and getting involved in helping people 1 think she felt a lot 
better; and just getting out in the community she met people who could put her in 
contact with different organisations. 


Achievement of high levels of 
engagement in wider 
community 


she’s involved with the school and she goes and helps with the lunches and things 
at school; so she’s getting involved, ‘cause she has a six-year-old child. . . . and she 
was showing me some handouts they had given her, she didn’t quite understand 
them and would I tell her what they meant, you know, about participating in the 
tuck shop and things like that. So she was willing to do that and pitching in and 
that was good. 


Greater engagement with social 
activities and expectations 


So first I meet people and I have to start to speak to them in different language, so 
people from different cultures. I . . . used to listen their different accents, 
sometimes very difficult. ... We started to meet even with some people . . . after 
community centre, go together to park with kids. So we started to have a social life 
with the group of people because many of them in that time got children, so we’ve 
got, you know, the meeting for something. So we meet regularly, during holidays, 
for example, even if here wasn’t any classes. (S) 


Improved personal 
competence; engaging with 
other beyond immediate 
learning environment 


One student, she’s about 55 and has a slight mental health issue and while she was 
doing the [LLN } programs her demeanour changed, her confidence changed, her 
dress changed; she was actually quite proud of her achievements and quite proud 
that she was able to stand up in the community and be noticed as being a good 
person rather than as just a nuisance value that she had been before. 


Greater engagement; improved 
body language and confidence 


We certainly hear again of lots of things, groups that people have joined and taken 
part in fitness programs, bush-walking, women who then started to cook from 
recipes and things like that and share those things as a network, whereas before 
they wouldn’t even share talking about a recipe because somebody might ask them 
for a copy of it, so they’d withdraw from that situation . . . whereas now they don’t 
feel they’re going to be caught out so they’re more happy to join into situations and 
become part of a network. 


Greater engagement in 
community activities 


The overwhelmingly consistent response here is that participants reported experiencing greater 
engagement in activities involving others and the community. Some suggested this was a result 
of personal competence that arose through participating in the programs. However, overall the 
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consistent claim was that there was a higher level of engagement in the community for these 
individuals arising from their participation in the non-accredited literacy and numeracy program. 

Summary 

What the examples in Tables 2-8 particularly show are the extent of the inter-relatedness of the 
seven proposed bases for identifying the wider benefits of learning, and the recurrence of certain 
general outcomes: (i) improved personal capacity and competence; (ii) empowerment, 
responsibility and autonomy; (iii) greater engagement with others and in the community, and (iv) 
a new capacity in learners for perceiving the world and their place in it. Underpinning every base 
is growth in learner confidence. 

It is evident from scanning the tables above that any attempt to monitor learner progress in non- 
accredited language, literacy and numeracy must take into account a wide range of possible 
indicators. This posed a challenge in developing an instrument that can meet local needs as well 
as be applicable across the diversity of programs. The final step in Phase 2 was therefore to 
obtain the views of the research partners about the extent to which it might be possible to 
develop indicators around each of the seven possible bases, with a view to ordering the perceived 
value of the bases. 
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Perceptions of indicators 



Table 9 summarises the perceptions of 46 paid and volunteer coordinators, paid teachers, and 
volunteer tutors at five sites, of the extent to which it might be possible to develop indicators, i.e. 
evidence or examples of changed student behaviour, for each of those seven factors. The 
interviewer clarified the meaning of each term before the ratings were given. 

There was strong support expressed in the interviews for the ‘confidence’ factor as being a 
valuable outcome of their participation in these programs. Respondents, generally, proposed 
there were numerous factors that would indicate growth in confidence and that these would be 
evident in their interactions with the learners. ‘Engagement with others’ and ‘social capital’ were 
generally regarded as strong factors, but there was doubt amongst some respondents that these 
could be ‘measured’ or observed because indicative activities tended to take place outside the 
learning situation. As one coordinator said: ‘We can only assess [social capital] as much as we 
know’. There appeared more opportunity to do that in neighbourhood centres, where there was 
opportunity for interaction between learners and with staff, than in one-to-one tutoring. 

There was similar ambivalence about ‘life trajectories’, with one tutor suggesting it sounded ‘a bit 
Californian’. Some respondents struggled with understanding the term ‘agency’, and for the Read 
Write Now! coordinators’ session, the term was changed to ‘Changes in pro-activity, belief in 
own empowerment, recognition of ability to influence others (sometimes called personal agency)’. 
However, even with that understanding, the perception generally was that it would be difficult to 
identify concrete and easily validated indicators. This reaction was even stronger for ‘personal 
growth’ which was described by one coordinator as an internal change and therefore far less 
visible that growth in self-confidence for which it was suggested there were observable 
behavioural changes. On the other hand, one person with significant experience in community 
settings believed changes in all the factors could be identified by developing appropriate 
‘scenarios’ to which the learners could respond. 

On the basis of the ratings given by the 46 respondents, as shown in Table 9, and the comments 
made in the interviews, as illustrated in Tables 2-8, the researchers decided to focus on the first 
five bases listed in Table 9, as a foundation for developing draft tools for monitoring progress, 
i.e.: 



• Self-confidence & personal competence 

• Engagement with others 

• Attitudes to learning 

• Life trajectories 

• Social capital 
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Table 9: Perceptions of coordinators, teachers & tutors of the extent to which indicators of learning outcomes can be identified for selected factors (Scale = 0-5) 



Factor 


A 


A1 


A2 


A3 


A4 


A5 


A6 


A7 


HI 


H2 


PR1 


PR2 


PR3 


PR4 


PR5 


PR6 


PR7 


Mean 

( n= 16) 


RWN 

(n=30) 


Overall 

Mean 

(n=46) 


Self-confidence 
& personal 
competence 




4 


5 


4 


5 


5 


5 


4 


4 


4 


5 


4.5 


5 


5 


4 


4 


4 


4.47 


4.78 


4.62 


Engagement 
with others 




4 


5 


5 


4 


4.5 


5 


3 


4 


2 


1.5 


4 


5 


5 


4 


3.5 


5 


4.03 


4.23 


4.13 


Attitudes to 
learning 




3 


3 


3 


4 


4 


5 


5 


4 


5 


3 


3 


3 


3 


3 


3 


3 


3.56 


4.23 


3.89 


Life trajectories 




3 


4 


2 


5 


3 


5 


4 


4 


3 


3 


4 


4 


4 


3 


0 


3 


3.37 


4.23 


3.80 


Social capital 




3 


4 


4 


4 


5 


5 


4 


4 


5 


0 


4.5 


4 


4 


4 


3 


5 


3.90 


3.60 


3.75 


Agency/ 

pro-activity 




4 


3 


2 


4 


3 


5 


3 


4 


2 


4 


3 


4 


4 


4 


3 


4 


3.50 


2.93 


3.21 


Personal 

growth/personal 




3 


4 


4 


5 


3 


5 


3 


4 


4 


3 


2 


2 


3 


1 


1 


3 


3.12 


3.33 


3.22 



change 

A Aberfoyle H Hackham West PR Pine Rivers N Northam/Toodyay P Perth RWN Read Write Now! coordinators conference 



The scale for the responses ranged from 0 (Don’t know) and 1 (Strongly disagree) to 5 (Strongly agree). At the sixth site, Glenroy Neighbourhood Learning Centre, 
which was the first site visited, the teachers and tutors were not asked for ratings in the initial interviews, but the coordinator subsequendy met with four teachers 
and provided written feedback which has been incorporated in the discussion below. One of the tutors (shown above without ratings) expressed the view that the 
whole approach was inappropriate for this level of learner assistance. 




Five bases were selected in order to allow for the number of themes that emerged from the 
interview data, as discussed above, i.e. growth in learner confidence, leading to improved personal 
capacity and competence, empowerment, responsibility and autonomy, greater engagement with 
others and in the community, a new capacity in learners for perceiving the world and their place 
in it. It was also felt that identifying a number of possible bases would ensure that the partner 
organisations were not overly restricted in their choices of wider learning outcomes when 
attempting to monitor progress. At the end of Phase 2, the tentative portfolio of instruments was 
beginning to take shape. 
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Collaborative development of 

portfolio 



In Phase 3 of the Action Research process, Dr Dymock and Associate Professor Billett worked 
with the Coordinators at the six sites to develop the ‘tentative portfolio’ into a ‘working portfolio’ 
that would become the basis for trialling in Phase 4. The first step in this phase was to survey the 
partners to canvass their views on a range of possible monitoring instruments. 



Examples of assessment tools 

Attached to Progress Report 1 were nine appendices which provided examples of tools used in 
assessing a range of ‘soft outcomes’, not all in the literacy and numeracy field. Dewson et al 
(2000:2) described soft outcomes as: ‘outcomes from training, support or guidance interventions, 
which unlike hard outcomes, such as qualifications and jobs, cannot be measured directly or 
tangibly’. 

In the discussions with partners, these tools were characterised as simply examples that had been 
found during the literature review and had been selected in order to provide a range of types of 
instruments that could be used, rather than necessarily being under serious consideration for 
adoption. Table 10 shows the responses by coordinators to the nine examples of ‘assessment’ 
tools. They were asked to provide their views on the potential usefulness of each instrument as a 
tool for determining wider outcomes of learning in non- accredited language, literacy and 
numeracy, on a scale of 1 (not useful) to 5 (very useful). 

On the basis of the respondents’ ratings and comments in the interviews, it was decided to focus 
on the first five of the monitoring tools listed in Table 10. As with the bases for identifying wider 
benefits of learning, discussed in the section above, it was decided that five tools would provide 
the partners with flexibility in determining which tools might be most appropriate for the learners 
in their particular programs. 
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Table 10: Responses of coordinators, teachers & tutors to examples of 



Tool / 
Model 



A1 A2 HI PR PR PR PR 

12 3 4 



App 1 Personal, social, 4 2 2 5 5 5 5 

economic 

App 6 Confidence grid 4 5 5 4 4 5 4 

App 4 Learner success 5 3 2 2 3 2 2 

indicators 

App 5 Achievement criteria 1 3 4 4 4 5 5 

App 3 Soft outcomes profile 2 4 5 1 1 3 2 

App 8 Personal development 4.5 4 2 3 3 4 4 

plan 

App 2 Effects of learning 4 4 4 1 1 2 2 

matrix 



App 7 Assessment profile 1 1 2 1 1 2 1 

A Aberfoyle H Hackham West PR Pine Rivers 



tools (Scale 1-5) 



PR 

5 


PR 

6 


PR 

7 


N1 


N2 


N3 


PI 


P2 


P3 


P4 


Mean 

(n=17) 


RWN 

(n=31) 


Overall 

Mean 


5 


5 


5 


4 


5 


5 


4 


4 


4 


4 


4.29 


3.95 


4.12 


4 


4 


4 


4 


4 


3 


5 


3 


4 


3 


4.06 


3.90 


3.98 


2 


2 


2 


4 


5 


5 


4 


4 


1 


4 


3.06 


4.26 


3.66 


4 


4 


4 


4.5 


3 


2.5 


3 


2 


4 


5 


3.65 


2.85 


3.25 


2 


2 


2 


5 


5 


5 


4 


2.5 


1.5 


4 


3.00 


2.97 


2.98 


4 


3 


2 


4 


3 


3 


3 


1 


3 


2 


3.09 


2.03 


2.56 


2 


1 


1 


2.5 


1 


1 


2 


2 


4.5 


5 


2.35 


3.00 


2.67 


1 


1 


1 


4 


2 


2 


4 


1 


1 


1 


1.59 


1.61 


1.60 



N Northam/Toodyay P Perth RWN Read Write Now! coordinators conference 




Based on the responses summarised in Tables 2-8, along with 9 and 10, the researchers 
ultimately developed six trial instruments, A to F: 

A: Changes in confidence 

B: Learner outcomes: personal, social, economic 
C: Individual learner profile 
D: Learner indicators of success 
E: Criteria for achievement 
F: Skills and wider outcomes 

Each of instruments A to E is an adaptation of the five instruments rated highest by participants 
during the Phase 2 consultations, and incorporate the ratings of the list of bases around which 
indicators might be developed ; Instrument F is a composite tool developed by the researchers 
from their review of research and the interview responses. The intention was to provide a variety 
of instruments that might be utilised to meet local requirements Also, a key criterion for selection 
and adaptation was that the instrument had to be capable of making some assessment of 
‘progress’, however the latter was defined. Where appropriate, the wording has been modified in 
the draft instruments using suggestions about outcomes and indicators made during Phase 2. 

In the original proposal, it was intended that Phase 3 would involve negotiations with the 
coordinators at the six sites, in order to develop a set of instruments that all partners would be 
willing to trial. However, the consultations in Phase 2 were much more extensive, and therefore 
more productive, than anticipated (with input from a total of 93 coordinators, teachers, tutors 
and learners instead of the 30 envisaged in the original proposal), that the decision was made to 
vary the approach. 
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Partner profiles 



Aberfoyle Community Centre, South Australia 

The Aberfoyle Community Centre’s LLN program, funded by ACE and Community Benefit SA, 
provides individual, self-paced learning opportunities in all areas of literacy. Currently, the Centre 
caters for 20 students with 20 tutors, providing one to one tuition. The delivery style is realistic, 
informal, non-threatening and inexpensive with proven positive outcomes. 

Attendance remains high because of new enrolments, with previous participants gaining the 
confidence to access the next step in educational or vocational pathways. Referrals are 
predominantly word of mouth from past students. This travels across significant distances, 
including three council areas to enable students to access one- to-one learning. 

The program has provided both social and educational activities, encouraging community 
participation. A number of our students have begun volunteering at the centre, mentoring others 
through cultural groups and becoming more confident community leaders. 

Participants are individually assessed at the first session, by the educator, to analyse language and 
literacy needs and after, matched with volunteer tutors. Informal assessment is continual 
throughout the sessions as tutor and the educator observe levels of participation and confidence 
in addition to language and literacy competency. Student and tutor feedback are regularly required 
to determine appropriate goals, which are readdressed when necessary. The Community 
Development Officer evaluates the Centre Project, records additional data, referral details, 
participation and internal and external pathways. 

Program delivery is one to one, small conversation and grammar groups with a facilitator, and 
English through cooking; a student/ tutor driven class with the educator’s guidance. Computer 
access is available and encouraged during all literacy sessions. 

The Aberfoyle LLN program has not only benefited students in concrete, measurable levels of 
competence but has impacted on their daily lives, creating opportunities for self development 
through education in practical, meaningful and diverse situations. Increased self- esteem, building 
relationships and extending social networks and friendship groups are all benefits of participating 
in the program in addition to increased employability and learning opportunities. 

Through the partnerships of students/ tutors and educator as well as extended partnerships with 
English Language Services and TAFE SA, the Aberfoyle Community Centre strives to meet the 
needs of all participants making learning appropriate, enjoyable and sought after. 
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Glenroy Neighbourhood Learning Centre, Victoria 

Glenroy Neighbourhood Learning Centre had its beginnings in 1978. In that year, staff at 
Broadmeadows Community Health Centre found that many of their clients needed literacy skills. 
So they requested that the Council of Adult Education provide someone to train volunteer 
literacy tutors for an adult literacy program in Glenroy. 

The adult literacy program run by volunteers and set up in the Glenroy Library was one of the 
first in the country. The literacy program continued in the library on small grants as a voluntary 
learning group until a part time paid coordinator was appointed in 1980. 



Name changes 

In 27 years of operation, the organisation has grown and changed in many ways. It has undergone 
a number of name changes: 

The Northern Districts Literacy Group: Glenroy Program [1978-1979] 

The Glenroy Adult Literacy Group [1979-1990] 

Glenroy Adult Literacy and Community Learning Centre [1990-1997] 

Learning North West Inc [1997-2000] 

Glenroy Neighbourhood Learning Centre: [2000 to date] 

We are proud to be a not for profit organisation committed to working with and for people in 
our communities. 



Committee of management 

As an incorporated body, our overall direction and planning is overseen by an elected Committee 
of Management . The Committee members are volunteers from the community and from the 
organisation. They meet monthly. 



Change of homes 

By 1989 the organisation was successfully receiving funding from state, federal and local 
governments. It became necessary to move from the Glenroy Library to 6 Hartington Street and 
then to our current premises, 5 Nelson Street, in 1989. We are now looking for a new home, 
nearby in Glenroy. We want a place that is more accessible, and where we can combine our 
education program with neighbourhood house work. Moreland City Council’s has made a 
commitment to building a new ‘neighbourhood facility’ in Glenroy. 

Our programs 

Since the early 1980’s, we have submitted for government funds each year and are considered to 
be a leading organisation in adult literacy and basic education, and in conducting programs for 
early school leavers. We also have a large and expanding English as a second language program. 
In 2003 as part of the NAMEP Consortium, we began offering D1MIA funded ESL classes for 
newly arrived migrants and refugees. 

Our well equipped computer facility is networked and online. As well as computer training, we 
offer computer assisted language and literacy learning, and on-line learning. 
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In 1999, the organisation became a neighbourhood house. This extended our community 
development work and built on our commitment to harmony in diversity. Our neighbourhood 
house programs such as yoga, volunteer training, the African women’s group, health and fitness 
programs, and mosaic are expanding. 

We plan to continue offering a strong program of activities and good adult education and training 
opportunities for adults and young people. In these ways and for many years to come we will be 
part of Glenroy, serving the local community and contributing to its future. 



Non-accredited literacy programs 
Literacy and Road Rules Women’s Circus 
Story Group 

Women’s ESL at Fawkner Community House 

Our CGEA classes have a small number of non-accredited students in them 
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Hackham West Community Centre, South Australia 

The Hackham West Community Centre, South Australia, and the various groups and facilities 
that operate from the premises offer the community assistance, training and support. Services 
and facilities include: 

A Hall and Room Hire 
A Everyday Lunch Cafe 
A Food Co-Op 

A Children’s & Youth Programs 
A Play Groups & Free Creche 
A Men’s and Women’s Groups 
A Adult Community Education 
A Parenting and Self Development Courses 
A Seniors Groups 
A Adult Learning Support 
■v" Tax Help, Counselling and JP available 

In Adult Community Education (ACE), short courses include: 

A Computing Classes 
A Basic Computer 
A Internet/Email 
A Publisher 
A Word Processing 
A Arts & crafts 
A Gardening Course 
A Holistic health 
A Women’s Self Defence 

The Adult Learning Support Program provides 1:1 tuition in the following areas: 

A Literacy 
A Numeracy 
A Help with study 
A English conversation 
A Basic computing/internet 
A Drivers’ permit instruction 

Adult literacy tutors, who are all volunteers, meet with their students every Wednesday from 
9.30am to 1 1.30am in the hall at the Centre. There is a short break for morning tea. 
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Pine Rivers Neighbourhood Centre, Queensland 

The Neighbourhood Centre has been a part of the Pine Rivers Shire area for almost 20 years and 
has been successfully providing assistance with adult literacy for over 6 years. Through our 
program we have assisted students into work and further study. We also now have a student who 
had now successfully completed tutor training and has begun working with her own student. 
Through our experience we have identified the need to offer literacy support to all groups within 
the community. Our current students come from all the targeted areas and reflect the diversity of 
needs in the shire. 

Customised learning plans will be developed with each student participant, tailored to suit the 
needs of the individual. Such response is formulated after individual assessments are conducted, 
to determine current literacy numeracy skills level, and with due consideration of current skills, 
personal, social and career goals and learning style/ s and by listening to what the student desires 
most assistance with eg. Gaining a drivers licence, opening a bank account, filling out forms etc. 
or by what the student enjoys eg karaoke, cars, cooking. 

By responding to individuals needs in meaningful and relevant ways keeps ‘students’ motivated 
and benefits are felt immediately. After results are achieved, higher goals of training for gainful 
employment that were previously far out of reach for many people, become more attainable. 

We currendy have 20 matched tutors and students, our relationships with both extend over the 
12 month funding period and some have been maintained for 3 years. The Pine Rivers 
Neighbourhood Centre has established its Adult Literacy program within the local community. 
We have established referral networks and work closely with job referral agencies with the shire. 
Our ability to attract and maintain volunteers is a stand out feature of the program. This is done 
through professional coordination of the project, with support and training opportunities for 
tutors. We intend for this successful program to continue. 

The Pine Rivers Neighbourhood Centre has established long term volunteers who assist in 
community capacity building, share resources, knowledge, and wisdom and mentor other tutors. 

We are currently seeking funding under the Skilling Queenslanders for Work , DEIR program. 
Previously, our programs have been funded under Community literacy, Training In Communities 
through the Department of employment and Training. 
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Read Write Now! Adult Literacy Service, Western Australia 

Read Write Now! provides a literacy service to adults in the community with the support of 
almost 800 volunteer tutors spread throughout Western Australia. Central TAFE College based 
in the Northbridge business district hosts the 3.2 FTE staff and generously gives in-kind support 
to the work of the unit far beyond the local boundaries and into 26 regions, both metropolitan 
and regional 

Each region is coordinated by a small team of volunteer Coordinators who support the tutors 
and students to achieve their goals. The volunteer tutors complete an accredited training course 
(Course in Tutoring an Individual Adult in Literacy and Numeracy) before working on a one-to- 
one basis to assist the adult student with his/her immediate literacy needs. ESL students are 
welcome provided they have the basics of spoken English and can communicate their needs. 
There is no set curriculum for the students. The tutor and student work together to develop a 
learning plan to meet the needs of the individual student. Tutoring takes place for 1-2 hours per 
week in informal venues such as community centres and libraries. 



Statistics for 2006 at a glance 

(these are typical outcomes for the Program) 

A Number of active volunteer tutors — average 800 over the year 
A Metropolitan tutor training — 6 courses were run over 24 Saturdays 
A Regional tutor training — 10 weekend courses were run over Saturday and Sunday 
A Tutor Conferences — 2 were held on Saturdays — total 320 tutors attended 
A Coordinator Training — undertaken at bi-monthly meetings or in situ. 

A Number new tutors trained — 144 
A Number of students assisted — 807 

A Note: tutor and student numbers approx 20% lower than in previous years. Informal feedback indicates this is 
a result of high employment rates in WA. 

A Approx one-third students either gained or maintained employment as a result of tutoring 

A Approx one-third of students were enrolled in further studies and required the assistance of 
the tutor to cope with the literacy demands of their course materials 

A Approx one-third of students were seeking help with personal goals such as helping children 
with reading/homework, or to participate in their local community or to gain more personal 
independence. 

A Approx one-sixth of the students self identified as having a disability 
A Almost half of the students were from non-English speaking backgrounds 

Funding 

A The service is funded by the Department of Education and Training and has been in 
operation for 30 years. 

A The 26 regional groups share $40,000 to cover teaching resources, phone, internet, postage 
and other local operating expenses. 

A The central office has an operating budget of $55,000 for training, travel, printing, 
communications etc. 
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Partners’ reports on trial of 

instruments 



The participants in the workshop were: Tamara Angus, Pine Rivers Neighbourhood Centre, 
Lawnton, Queensland; Jacinta Agostinelli, Glenroy Neighbourhood Learning Centre, Victoria; 
John Pryer, Hackham West Community Centre, and Debbie Jeanes, Aberfoyle Community 
Centre, South Australia; Marcia Barclay and Pamela Thurbon, Read Write Now!, Western 
Australia, and Darryl Dymock, Griffith University, Brisbane. There was an apology from Stephen 
Billett, Griffith University, who was overseas at the time. A summary of each partner’s report on 
trialling the instruments is presented below. 

Aberfoyle Community Centre 

The Adult Literacy Coordinator at Aberfoyle Community Centre in South Australia, Debbie 
Jeanes, wanted the tutors to have ownership of the process, so she left it up to them as to which 
instruments they chose and how they delivered them. She copied all of the instruments, A to F 
and gave them to all of the tutors and met with them for an introductory talk, and gave them 
more background as well on what the aims were. Ten of the twenty tutors responded, and usually 
it was the ones who had been involved in the interviews in the data collection phase in the first 
place because they felt comfortable with the instruments. 

Three tutors worked with their student in filling out all or most of the instruments over the 
period available and gave feedback. Seven tutors more or less critically looked at the instruments, 
mainly themselves but in conjunction with the student 

For the volunteer tutors at Aberfoyle Community Centre, Trial instrument D was an outright 
winner. Debbie Jeanes put this down to the fact many of the clients are focused on English as a 
second language. So the language used in the instruments was very important and the tutors did 
not want it to be seen as patronising to the students, or too taxing or confusing. A lot of the 
language they found confusing in other areas. Debbie thought that with instrument D ‘ticking the 
boxes’ probably made the students feel more comfortable, because they did not have to actually 
physically write anything. For this group, instrument F had much more negative feedback - they 
found it too jargonistic, too verbose and very difficult. 

The tutors thought instruments B and C were quite similar in content. Debbie observed that 
although instrument D would be the instrument most readily acceptable by tutors and students at 
the Centre in their current situation, that might change if the nature of the clients changed. She 
said: 



We do seem to have a period of change with clients. Originally when I started at the 
Centre it was more literacy but we have a partnership — unofficial partnership — between 
the medical centre, a lot of the Japanese doctors work there, so their wives come to us. So 
a lot of our students are from Japan, etc. So if the need for doctors changed then . . . that 
instrument might change too. 

The suggestion with this instrument was that they would use it initially, perhaps at the initial 
assessment and then maybe annually because that’d be enough time in between for the students 
to monitor their progress and probably forget that they’ve done it in the first place. 
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Debbie Jeanes’ overall assessment of the process was: ‘It was great. Ellen [Jezierski, the 
Community Development Officer] and I were really pleased to be involved in the project because 
we’ve always been looking for something to measure confidence, to measure the success our 
students have.’ 

Glenroy Neighbourhood Learning Centre 

The Adult Literacy Coordinator at Glenroy Neighbourhood Centre in Victoria, Jacinta 
Agostinelli, gave a copy of each instrument to the teachers with a covering note and asked them 
to select the tool they thought would best suit their class. The short period of the trial made it too 
difficult for most tutors to apply the instrument twice. However one of the teachers taught a 
specific skill in the four-week period and then got the students to apply the tool to the learning of 
that skill, which apparently worked quite well. 

The teacher from Literacy and Road Rules selected instrument B because he thought it reflected 
the multi purposes of his class and because it looked simple for him and the students to try to fill 
out together. He said the students could not manage the literacy required by the instrument so 
they did it by discussion, and then the students selected what they would write in the comments. 
The teacher was surprised that some students responded so well, and thought it was a good tool 
because it got the students talking about their goals. 

The tutor for the Women’s English as a Second Language (ESL) class selected instruments C and 
D, and also created her own student self-assessment instrument, because she found that the other 
instruments too complex for the students’ language level. With instruments C and D, she said she 
still needed to do a fair bit of explaining of the concepts and goals. But once the students did 
grasp the concepts they could answer the questions easily; they knew exacdy what the questions 
were asking. The tutor was quite excited by the student responses to the tutor-developed 
instrument, because she could see that outcomes did improve over the time of the trial. 

The Story Group selected instrument C because it had a greater focus on reading, writing and 
discussion skills, which is really what the group is about, rather than on employability. Most of 
them are around retirement age, so they are not interested in employment, at least not paid 
employment. Given the profile, some of the questions were still irrelevant, such as personal 
growth, which was not seen as appropriate for people who are quite established in their life. 

The African Women’s Sewing Class selected instrument F because it tended to have a focus on 
skills development which they saw as appropriate for what they were doing, although the class 
also serves a big social need for the African community. 

There was also one non- accredited student in the General Certificate of Education for Adults 
(GCEA) class, and she selected instrument E. She liked the achievement focus because according 
to Jacinta Agostinelli that is how the student describes her learning - she likes to see what she is 
achieving and that she is achieving. The student ticked the things she came into class for and left 
the other ones out. One question raised for instrument E was where to go next, since there is no 
provision for showing progress. 

On some of the instruments, there were too many words or words difficult for a second language 
speaker to understand. On the other hand, according to Jacinta Agostinelli: ‘Teachers were 
enthusiastic about having a measure for achievement not couched in the language of training 
packages and competency based training - it was just a bit of fresh air for them.’ 

Several of the workshop participants mentioned that one outcome of the trials had been positive 
experiences for the students in using the instruments, regardless of the content. For example, the 
‘Road Rules’ course tutor had said the activity of actually completing the instrument raised much 
discussion in areas that might not have been covered, such as respecting others’ rights, life goals, 
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and the rules we live by. He said it had broadened the class and he had found it a very interesting 
and worthwhile activity. In the Women’s ESL class, the tutor though the instrument enabled the 
students ‘to begin to see the connection between what they do in class and how it relates to life 
outside and how they transfer that to everyday living’. 

Hackham West Community Centre 

The Coordinator of Hackham West Community Centre, John Pryer, left it up to the tutors to 
choose the instruments they preferred, and the choices were: instrument C: four, instrument E: 
two, instrument F: two, and there was one student who tried all six. 

The general consensus about Tools A, B and D was that they were a little complicated for some 
tutors, and hence for the learners. Overall the preferred instrument for both tutor and student 
was instrument C. The overall tutor feedback to the use of the instruments was good - they 
welcomed the chance to record progress over and above the sheets that they already fill in. 

Some of the tutors said that there were occasional inconsistencies insofar as measuring across a 
period of time. For example, one tutor noted that on one instrument a student had said that he or 
she felt more confident and yet later on they said they felt about the same. Other issues were 
irregular attendance by both tutors and students, and some instruments were filled in on one 
session only, others across a series of sessions. Some tutors ‘forgot’ to use the instruments, whilst 
others used the different instruments at different times. 

One question raised by tutors was whether students were filling in the instruments with incorrect 
or untrue responses that they feel look good on paper - as if, as John Pryer said: ‘they’re on their 
best behaviour, being assessed or examined’. On the other hand, one tutor who initially thought 
the instruments were a waste of time reversed her opinion after trying one out and getting good 
feedback from the student. The coordinator thought that the need for finding ways of measuring 
social capital was a major outcome from the trial. 



Pine Rivers Neighbourhood Centre 

The Team Leader - Community Learning at Pine Rivers Community Centre, Tamara Angus, sent 
the instruments to six volunteer tutors who had been with the adult literacy program for some 
time. As per the suggestion in the covering letter from Griffith University, she suggested they 
choose two to use with students, but all six chose to use all the instruments with their students. 
Only one tutor was able to apply the instruments at the beginning and end of the trial period, and 
reported that the student was much more comfortable doing it a second time. 

Some of the negative feedback from the tutors in the trial was that it was a ‘tough exercise’, that it 
was ‘a bit of a challenge’. Some said that some questions were obscure and a bit convoluted, and 
some of the tutors struggled with the terms, ‘confidence’ and ‘self-esteem’ and how to explain 
those concepts to a student. Another issue in relation to not understanding the terminology was 
the diversity of backgrounds and educational levels of the tutors themselves — quite a few of 
them are teachers but quite a few of them are not and come from different backgrounds. Some 
saw the concepts as ‘university-fied’, and ‘academic’. Some of the students were scared of filling 
in the instruments, even in consultation with the tutors, and found it very daunting. 

On the other hand, one tutor reported that ‘the student found it was really worthwhile 
completing it and that they felt really at ease and good with doing it’. Another said the instrument 
provided the student with specific learning outcomes that he could recognise and that he was able 
to recognise his own improvements — ‘that’s where I’ve gone and that’s where I’m at on some of 
those scales’, as summed up by the team leader. 
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It was suggested that for the different target groups the questions could be worded differendy. 
Tamara Angus also felt that the tutors needed more support and more initial information about 
the instruments, and attributed this lack to a changeover of literacy staff at the Centre at the time 
of the trial. A challenge was to cater for the big variance between students, and to draw into one 
response how the questions are asked and how the answers are written. 

The majority of the tutors picked the tools A and C as being most relevant because, they said, the 
items could be applied in the real world. Tool C was seen as a good self-awareness tool. Most of 
the tutors understand the need for confidence building and self-esteem, and that tools A and C 
were seen as being able to measure improvement in confidence and the main feedback was that 
those two instruments covered the relevant areas ‘really, really simply’. 

One suggestion arising from the trial was that the Centre might adapt its initial assessment of 
students by including provision for rating students’ improvement in e.g. self-esteem, over the 
period of the tutoring. One tutors suggested complementing the initial assessment tasks with 
asking the students to rate themselves on a scale in relation to their confidence with those tasks. 
The coordinator suggested that this would have to be in relation to the student’s literacy levels 
too because of the range of students, from ‘university students from Chile doing their PhD who 
just need the English practice, as opposed to someone who’s come and who doesn’t know their 
ABC’. 

As in the Glenroy program, Tamara Angus said there was evidence that the students gained 
confidence from the process of completing the instruments: “Wow, 1 filled out a instrument”. 

She said it helped the students think about the progress they had made and that the tutors could 
reinforce that. At the same time, she thought it helped the tutors ‘re-examine the what, why and 
how of their own tutoring’. 

Read Write Now! 

The Manager of the Read Write Now! program, Marcia Barclay, the Coordinator of Operations, 
Pamela Thurbon and one of the regional volunteer coordinators in Western Australia examined 
the six the instruments and chose C and E as the two most appropriate for the totors and 
students in the program. Instrument C was chosen because of its simplicity — it was short and 
simple and appeared easy to fill out. Instrument E was seen as more thorough, and more specific 
with all the outcomes. However changes were made to both instruments before they were sent to 
tutors, including wording where it was felt that it was too hard for students to understand. In 
instrument C, the wording was modified to make sure that the tutor and the student completed it 
together, so that the tutor did not complete it independently and guess at where their student was 
at — the intention was for the student to take part and say where they felt confident: ‘We felt they 
needed to be empowered to give the answers on this and [that] the tutor might . . . explain what 
the questions were but it needed to be the student’s answers, not the tutors’. 

Also added to both instruments was provision for the student to sign at the bottom, with the 
student’s name and signature first to give them a sense of empowerment, that ‘this is my 
instrument that I’ve . . . taken part in’, as Pam Thurbon said. The column headings on instrument 
C were also changed slightly to ones that were seen as easier to understand, and the provision for 
a total score for the assessment was deleted. 

Trial instrument E was also revised, including the addition of a column on the left hand side 
which allowed students to indicate at the initial session what they wanted to achieve. Columns 
were also added on the right hand side to allow for progression: ‘same’, ‘progressing’ and 
‘achieved’, rather than the single ‘achieved’ — so students and tutors could see some sort of 
progression. In the trial, because most tutors and students were already advanced in their 
tutoring, the left hand ‘goal-setting’ column seemed irrelevant and their was some confusion 
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evident in the responses. However it was felt the instrument would work if it used initially if the 
tutors were given training in using it. 

Both versions were emailed out state-wide to some 300 tutors. Despite the instructions in a cover 
letter, around half the instruments were completed and returned without any comment on their 
use or the reaction of students. Pam Thurbon followed up a number of those in order to obtain 
more feedback. Another limitation was that because the instruments were sent out on email as 
attachments, the tutors were responsible for printing them off and putting them together and 
some of them thought that all the instruments were combined in one. This misunderstanding led 
to some comments that the questions were very repetitive. 

34 responses were returned without comments, and 49 responses were returned with comments, 
including some from those recontacted in the follow-up. There were a number of positives and a 
number of negatives with both instruments, but language was a big issue. About half the students 
in the Read Write Now! program are English as a second Language (ESL) students, and there was 
a very strong response that the language was too difficult for those students unless they were 
quite advanced. It took a long time for the tutors to explain the words that the ESL students 
couldn’t understand. Also, one tutor suggested that in some cultures people often find it hard to 
give a negative comment and so she was not too sure whether those responses were entirely 
accurate. 

In considering the responses, the RWN team initially considered either making the language 
much simpler or having separate instruments for ESL students and native English speakers.. It 
was clear to them that needed a simpler instrument with simpler language. Finally they decided to 
re-develop instrument E for their purposes. The aim of the new instrument, a draft of which is 
attached as Appendix 3, is to use it to set up the learning goals, and for the tutor and the student 
to use it as a reporting mechanism, so that combined with the instrument presently used to 
provide feedback to coordinators and to provide data for regular reports to the Western 
Australian government, it would help students to see the progress they had made. The intention 
is to help the students see the progress but also to keep the tutor focused on basing the lesson 
around the student’s goals. Marcia Barclay summed up the thinking behind the revised 
instrument: 

We based it largely on the feedback that . . . the other instruments were long and tedious, 
and given that our tutors and students only meet once a week for an hour and a half, some 
of them were finding that their whole sessions were taken up on a instrument. So what 
we’ve tried to do is to reduce it down to what we think the essentials within it, just within 
our program, but also too, as a way of giving the tutor recognition. 1 think currently our 
tutors are really hard on themselves, that they think unless their students turned out to be a 
Rhodes scholar then they haven’t done that good a job at tutoring, and I don’t think they 
acknowledge the soft outcomes. And so that’s why I was really keen about the project and 
. . . this new instrument of evaluation, which will be combined with our [current reporting 
instrument] . 

The two Read Write Now! representatives at the Adelaide workshop emphasised that the new 
instrument was still a draft and needed more thought before it was finalised, including how often 
it would be used in the tutoring process. 
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Letter to partners May 2007 



Dear [Partner] 

As promised, attached is a ‘portfolio’ of six possible instruments for measuring progress in non- 
accredited language literacy and numeracy, for you to trial with your tutors and students. These 
have been developed following interviews and discussions in four states arranged through the six 
partners with coordinators, teachers, tutors and students. 

Originally our plan was to send these instruments for your responses, and then for us to fine tune 
them before sending them to you again. However, given the extensive consultations that have 
taken place over the last two months, and the time constraints of the project, it seems more 
expeditious to send them to you as drafts for trialling, but with the proviso that you can amend 
these individually for ‘local’ use if you wish, as well as create your own instrument to add to the 
portfolio, if you prefer. 

Here is what we would like you to do: 

Consider which of these draft instruments, if any, might be appropriate for monitoring the 
progress of learners in your language, literacy and numeracy program. 

If necessary, feel free to modify the wording slightly to suit your own circumstances. You may 
also want to create a completely new instrument to test in your program, now that you have seen 
what these are like.* 

Try these instruments with as many tutors and students as possible between now and mid July. 

Document the feedback from tutors and students on the instruments you select for use, along 
with any changes you have made, and report on these at the workshop in Adelaide on 20 July. 

Among the questions you might address are: 

A How can these instruments best be used in your practice? 

A What are their relative strengths and weaknesses? 

A Are there any particular preferences for instruments, and if so, why? 

A If you had to select just two of these, which would they be and why? 

A Are there issues of monitoring progress in non- accredited language, literacy and numeracy 
which are not met by any of the instruments in the portfolio? 

A To what extent might it be possible to adopt one or more of these instruments for national 
use? If this is a feasible idea, how might that be done? 

*One suggestion made to us was the possibility of developing a number of scenarios against each of the 
possible bases for monitoring progress, e.g. for ‘Confidence’, the scenario might be: ‘You have been 
dropped in the middle of [city]. Would you be able to get home from there, and if so how?’ We think this 
notion is worth considering, but that the scenarios would best be developed locally rather than genetically 
by us, in order to meet individual needs. 

The original intention of this project was to see if it is possible to identify wider outcomes, i.e. 
those beyond the development of language, literacy and numeracy skills. Some of the attached 
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instruments maintain that focus, and might be used in conjunction with skills assessments you 
already use; some of them incorporate both skills and broader outcomes. 

Ideally, ‘progress’ is measured between two points, so it would be best if you could trial these 
over at least two sessions, say a month apart. However, if that is not possible within the time 
available, even a single session should help determine if an instrument is appropriate or not. 

Please contact us if you have any concerns or queries. We will share these where appropriate with 
the other partners in order to keep you all involved. 

We would appreciate it if you could acknowledge receipt of the portfolio, and perhaps give us 
some indication of how you are likely to trial the instruments over the next eight weeks. 

Darryl will give you a call within the next couple of weeks to see how you are going with 
implementing the trial. 

Sincerely 

Darryl Dymock and Stephen Billett 
Griffith University 
18 May 2007 
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Interview schedules - Pilot 



Monitoring progress in non-accredited 
adult language literacy and numeracy learning 

Questions for learners/ students 

1. How long have you been learning in this particular course/program? 

2. How many hours a week do you have with your tutor/ teacher/in the class? 

3. Why did you decide you needed to do this? Why did you seek help? [motivations, goals] 

4. In what ways are you making progress? 

5. Can you identify outcomes you are achieving beyond those directly associated with literacy and 
numeracy? 

6. How much is this literacy and numeracy course/ program a part of your life? 

For example, is it really important to you, or is it just another thing you do, or is somewhere 
between. Do you think about literacy and numeracy much when your not here? Do you do any 
work on it at home? Do you miss many sessions? [Explore for reasons.] 

7. Tell me a bit about how the teacher/ tutor tells you how you are going in the course/ program? 
What sort of feedback do you get? [If formal/ structured assessment]: what do you think of that? 

8. Do you get a chance to tell the teacher how you think you’re going? In what ways? What other 
sorts of things do you think a student could tell the teacher to let them know how the student is 
going? 

9. Since you started this course, do you think there have been any other changes in what you can 
do and how you feel about yourself? (general question) 

10. After discussion re responses to Q9, follow up specific areas as necessary: 

Do you think you’re more personally confident now than you were when you started the course? 
How can you tell? Why is that? 



Y/N 


Progress with personal confidence 







Has engaging in the course affected your relationships with other people — family, in the local 
community (shopping etc), workplace (if applicable). In what ways? How can you tell? Why is 
that? 



Y/N 


Progress with relationships with others 
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Do you think participating in the course has changed the way you think about yourself, for 
example, do you think you are more independent, able to do things by yourself that you couldn’t 
do before? How can you tell? Why is that? 



Y/N 


Progress with sense of self 







Has all this changed your view about what you would like to do with your life? More training? 
Work or better work? Attitude to life? In what ways have these changed? 



Y/N 


Progress with personal goals and attitudes 







Do you think you understand any better about how you learn? How you like to learn? How did 
this come about? 



Y/N 


Changes in conceptions of learning 







Are there any negative outcomes for you in doing this course/ program? (E.g. The way you see 
yourself, attitude to further education etc, financial cost) how did this come about? 



Y/N 


Unwelcome forms of progress 







11. One of the ideas we’re looking at with this project is whether we can show that a student is 
making progress in lots of ways, for example, that they’re growing in confidence, so that they’re 
able to do things they couldn’t do before, as well as reading and writing tasks. For example, how 
would you respond to each of the following: [would you prefer to read them or would you rather 
I read these out to you?] 



Benefit (progress) 
from LLN 
course/ program 


1 Not at all 


2 - Very 
little 


3 Somewhat 
- definitely 
helping a 
bit 


4 - Quite a 
bit 


5 Very 
much 


Improved Personal 
confidence 












Improved Relationships 
with others 












Improved sense of self 
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Benefit (progress) 
from LLN 
course/ program 


1 Not at all 


2 - Very 
little 


3 Somewhat 
- definitely 
helping a 
bit 


4 - Quite a 
bit 


5 Very 
much 


Improved personal 
goals and aspirations 












Improved contribution 
to the community or 
groups you’re a part of 












Improved 

understanding of how 
you learn 












Other '(please name) 












Other '(please name) 












Other '(please name) 





































12. Do you have any other comments or questions? 



Thank you very much for your time. 
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Questions for tutors/ teachers/coordinators 

1 . How long have you been involved in teaching adult language and literacy? 



2. Tell me a litde bit about your current teaching? (e.g. who, what, where, how?) 



3. What sorts of assessment do you use? 



4. To what extent do you think personal confidence is a factor in successful learning for learners 
in non-accredited learning? 

5. In the research summary we sent you prior to this interview, we identified seven possible bases 
for identifying if progress had been made, beyond the development of literacy and numeracy 
skills. Sometimes these are called the ‘wider benefits of learning’ or ‘soft outcomes’. 

We’d like to get your opinion on how useful you think each one of those is and whether you 
think there are indicators you could use to say that a student is making progress in that particular 
area: 

— degree by which self esteem and personal confidence have changed, and in what ways, as a 
result of participation in literacy program (i.e. Self-confidence and personal competence) 

— degree by which engagement with others has changed, and in what ways, as a result of 
participation in literacy program (e.g. family, workplace, teacher). (I.e. Engagement with others) 

— extent of change of attitude towards learning and future learning and ability to learn how to 
learn (i.e. Attitudes to learning) 

— degree by which individuals’ agency and pro-activity have changed, and in what ways, as a result 
of participation in literacy program (i.e. Agency/ pro-activity) 

— degree by which individuals’ life trajectories have changed, and in what ways, as a result of 
participation in literacy program (i.e. Life trajectories) 

— extent to which a learner perceives they have grown as a person as the result of engagement in 
a LLN program (i.e. Personal growth/personal change ) 

— extent of changes in community participation, involvement in networks. (I.e. Social Capital) 
What other benefits you have identified, if any? 

6. We also sent you as Appendices some examples of assessment tools that have been used with 
non-accredited learning, and we’d appreciate your thoughts on the potential value of each of 
those in monitoring progress in literacy and numeracy learning. 

(These aren’t intended to be necessarily what we want to end up with this in this project. They’re 
just a range of instruments we found in our research and your opinion will help decide whether 
any of them are worth considering.) 

To begin with, is there any one of these that appeals to you more than the others? If so, why? 
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Please tell me what you think of each of the others: 



Benefit (progress) 


1 Not useful 


2 - Very 


3- 


4 -Quite 


5 Very useful 






little use 


Somewhat 

useful 


useful 





Appendix 1: Outcomes for providers/stakeholders/funders & Outcomes experienced by the 
learner (Foster et al 1997) 



Utility (i.e. 1 - 5) 


Outcomes for providers & learners(why this ranking?) 







Appendix 2: Matrix classifying the effects of learning (Schuller et al 2004) 



Utility (i.e. 1 - 5) 


Classifying the effects of learning (why this ranking?) 







Appendix 3: Getting to know you (SOUL) (Anderson et al 2006) 



Utility (i.e. 1 - 5) 


Getting to know you (why this ranking?) 







Appendix 4: Indicators of success identified by students (Eldred 2002) 



Utility (i.e. 1 - 5) 


Student identified successes (why this ranking?) 







Appendix 5: Emergent criteria of achievement (Charnley and Jones 1978) 



Utility (i.e. 1 - 5) 


Emergent criteria (why this ranking?) 







Appendix 6: Catching confidence (Eldred et al 2004) 



Utility (i.e. 1 - 5) 


Catching confidence (why this ranking?) 
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Appendix 7: Assessment profile (note: this is an example of the wheel idea, based on rankings 
against indicators — the indicators would be different for L &N learning) (Department for Work 
and Pensions 2004) 



Utility (i.e. 1 - 5) 


Assessment profile (why this ranking?) 









Appendix 8 — Personal development plan (Department for Work and Pensions 2004) 



Utility (i.e. 1 - 5) 


Personal development plans (why this ranking?) 







Appendix 9: Soft indicators: Individual profiling (Department for Work and Pensions 2004) 



Utility (i.e. 1 - 5) 


Soft indicators (why this ranking?) 







Others 1 



Utility (i.e. 1 - 5) 


(why this ranking?) 







Other 2 



Utility (i.e. 1 - 5) 


( why this ranking?) 







7. Overall, What do you consider as the pros and cons of using indicators to monitor the 
progress of L & N learners beyond the growth of literacy and numeracy skills? 

Strengths 



Limitations or problems 



8. In what ways do you think it should be possible or desirable to develop a range of monitoring 
tools which would be acceptable nationally, or will these work best at a local level? 
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9. What might these tools look like? 



10. Any other suggestions or comments? 



Thank you very much for your time. 
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Interview schedules - Revised 



Monitoring progress in non-accredited 
adult language literacy and numeracy learning 



Name of partner organization: 



STUDENTS - MOTIVATION & PROGRESS 

I want to ask you some questions about your motivation for doing this course and your progress 
within it. 



1. First 
name 


2. 

Current 

course 

or 

program 
at this 
Centre? 


3. No. 
mths 
or yrs 
at this 
Centre? 


4. Why 
did you 
decide 
to 

enrol in 
this 

course? 

(What 

made 

you 

decide 

you 

needed 

to get 

help or 

support 

in this 

way?) 


5. 

What 
do you 
expect 
to get 
out of 
this 

course? 


6. Do you 
think you 
are making 
progress in 
achieving 
those 
purposes? 

No (1) 

Not sure (2) 

Yes (3) 


7. Why do 
you think 
that? 

(Follow-up: 
are there 
things you 
can do now 
that you 
couldn’t do 
before you 
started the 
course?) 


8. Tell me 
about the 
assessment 
tasks you 
do. How 
do you 
know how 
you are 
going in 
the 

course? 
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STUDENTS - OTHER LEARNING OUTCOMES 



I now want to ask you some questions about the things that you have learnt or other outcomes, 
which might be different from what you are supposed to learn 



First name 


9. Do you 
think there 
are other 
benefits, apart 
from 

language and 
literacy, that 
have come 
from doing 
this course? 


10. What do 
you think 
those 

benefits are? 


1 1 . How 
important are 
those other 
benefits for 
you? 

No importance (1) 
Low 

importance(2) 

Moderate 
importance(3 ) 

High importance 

( 4 ) 

Vety high 
importance (5) 


12. How do 
you think you 
learned those 
other 
benefits? 


13. What 
impact do you 
think the 
course had on 
bringing about 
those broader 
outcomes? 

None (1) 

Very little (2) 

Some (3) 

Quite a bit (4) 

Very much (5 
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STUDENTS - CONFIDENCE 



In this section, I'm going to ask you some questions about whether your personal confidence has 
changed as a result of your participation in the course 



First 

name 


14. 

Were 

you 

scared 

or 

anxious 
when 
you first 
came to 
this 

Centre? 
How do 
you feel 
about it 
now? 
(Explore 
for 

learning 

climate 

& 

teacher 

attitudes 

& 

support) 


15. Since 
you 

started this 
course, do 
you think 
your 

confidence 

has 

increased: 

None (1) 

Very 
little(2 ) 

Some (3) 

Quite a bit 

( 4 ) 

Very much 

( 5 ) 


16. Why is this? 
(If positive: Can 
you give me 
some examples 
of how this 

confidence 

shows, eg with 

family, 

shopping, 

doctor/ chemist, 

workplace, 

community) 


17. How 
important 
is this 
course to 
you? Is it: 

Not very 
important 
(1), 

Reasonably 
/ moderately 
important 

(2) , or Very 
important 

( 3 ) 


18. 

Why 
do you 
think 
of it in 
that 
way? 


19. Will 

you get a 
certificate 
at the end 
of this 
course? 

Is that 
important 
to you? 
Why or 
why not? 


20. 

What’s 
the next 
step for 
you after 
this? 

How 
confident 
are you 
that you 
will 

achieve 

your 

goal? 

Not 

confident 

(1) , a little 
confident 

(2) , quite 
or 

reasonably 

confident 

(3) , very 
confident 

( 4 ) . 

Discuss 
reasons + 
anything 
else 

student 
wants to 
talk about 
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STUDENTS - CONFIDENCE STATEMENTS 



21. Confidence statements 

I am going to show you and read out 10 statements about being confident, and I’d like you to tell 
me how you feel about each one of them. There are four possibilities: not confident (1), a little 
confident (2), quite confident or reasonably confident (3), very confident (4). I’ll ask you to pick one of those 
four for each of the statements. Please ask me if you don’t understand any of them. 

Student 



names 



Situation 


name 


name 


name 


name 


name 


name 


name 


name 


name 


1 . 1 am confident when 
meeting new people 




















2. 1 am confident 1 can 
learn from the course I 
am doing at this Centre 




















3. 1 am confident that I 
can apply what I learn 
here to my daily life 




















4. 1 am confident to 
speak in a group 




















5. 1 am confident to 
speak to a person I don’t 
know 




















6. 1 am confident when 
writing things down in 
English 




















7. 1 am confident that 1 
am good at some things 
and have valuable skills 




















8. 1 am confident of 
taking charge of my own 
affairs 




















9. 1 am confident in 
strange situations or 
strange places 




















10. I feel 1 am generally a 
confident person. 
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22. How much do you 
think the situations 
where you think you 
have developed some 
confidence (a litde, quite, 
very) are a result of doing 
this coursePIVftW (1), Very 
little (2), Some (3), Quite a 
bit (4), Very much (5) 
Discuss responses 





















23. Do you want to say anything else about your own learning or what you have got from this 
course? 
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Monitoring progress in non-accredited 
adult language literacy and numeracy learning 

COORDINATORS/TEACHERS/TUTORS - PROFILES 

Name of partner organization: 

The figures from this Profile will be used in aggregate with the other partners across the project 
in order to show the range and nature of the people who have contributed their views to the 
development of the ‘monitoring progress’ portfolio. Individual names will not be used in the 
report. 



1. First 
name 


2. Total 
No.years 
teaching 
(all/ any 
subjects) 


3. Total 
No.years 
teaching 
Lang, 

Lit 

and/ or 
Num 


4. No. yrs 
working 
with this 
organisation 


5. Role at this 
Centre & 
subject/ s taught 

(e.g.coordinator, 
teacher, tutor + 
subject/ s) 


6. 

Volunt- 
eer or 
paid? 


7. Any other 

relevant 

information 
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COORDINATORS/TEACHERS/TUTORS - OTHER LEARNING OUTCOMES 



8. Can you 9. What are 
identify other those 



learning 
benefits for 
your students, 
apart from 
language and 
literacy, that 
have resulted 
from this 
course? 



benefits? 



10. How 
important are 
these benefits 
for your 
students? 

No importance (1 ) 

Vow 

importance(2) 

Moderate 
importance(3 ) 

Nigh importance 

( 4 ) 

Very high 
importance (5) 



11. For each 
of these 
benefits can 
you identify 
how you 
believe 
students 
come to learn 
them? 



12. What 
impact do you 
think the 
course had on 
those broader 
outcomes? 

None (1) 

Very little (2) 

Some (3) 

Quite a bit (4) 

Very much (5) 
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COORDINATORS/TEACHERS/TUTORS - CONFIDENCE 



First 

name 


13. To what 
extent do you 
agree 

confidence is a 
factor for 
successful 
learning for 
learners in 
non-accredited 
LLN 

Learning? 

Don’t know (0), 
Strongly 
Disagree (1), 

Disagree (2), 
Neutral (3), 
Agree (4), 

Strongly agree 

( 5 )- 


14. Why 
do you 
believe 
that? [If 
positive: in 
what ways 
have you 
seen 

confidence 

expressed 

by 

students?] 


15. Do you 
think that 
the support 
given by the 
Centre and 
the attitudes 
of teachers 
or tutors 
influences 
student 
confidence? 
In what 
ways? 


16. Do 
you think 
it is 

possible to 

develop 

teaching 

strategies 

that will 

help 

develop 

students’ 

confidence 

and other 

personal 

outcomes 

as well as 

LLN 

skills? 

Please 

explain. 


17. Do 
you think 
students 
should be 
involved 
in 

assessing 
their own 
progress; 
why or 
why not? 


1 8. Can you 
see any 
disadvantages 
in trying to 
monitor 
students’ 
progress in 
personal 
development 
along with 
assessing 
their LLN 
skills? 
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COORDINATORS/TEACHERS/TUTORS - BASES FOR INDICATORS 

19. POSSIBLE BASES FOR IDENTIFYING WIDER BENEFITS OF LEARNING/SOFT 
OUTCOMES 

We are looking at whether it might be possible to develop some indicators for monitoring the 
wider benefits of learning, based on the summary of research literature sent to the Centre in 
advance of this interview. At the end of that paper we listed seven possible bases for such 
indicators. 

The question is: To what extent do you agree that each of these seven factors might provide a 
useful basis on which to develop some indicators, i.e. evidence of changed student behaviour? 
There are five possible responses: Don’t know (0), Strongly Disagree (1), Disagree (2), Neutral(3), Agree 
(4), Strongly agree (5). 

Please give reasons for your responses, including where possible examples of possible indicators. 



Teachers’ ratings (0-5) 



Factor 


name 


name 


name 


name 


name 


name 


name 


Self-confidence and personal competence 
(degree by which self esteem and 
personal confidence have changed, and in 
what ways, as a result of participation in 
LLN program) 
















Engagement with others (degree by 
which engagement with others has 
changed, and in what ways, as a result of 
participation in LLN program, e.g. 
family, workplace, teachers). 
















Attitudes to learning (extent of change of 
attitude towards learning and future 
learning and ability to learn how to learn 
as a result of participation in LLN 
program ) 
















Agency/ pro-activity (degree by which 
individuals’ agency and pro-activity have 
changed, and in what ways, as a result of 
participation in LLN program) 
















Life trajectories (degree by which 
individuals’ life trajectories, i.e. goals, 
ambitions, purposes, have changed, and 
in what ways, as a result of participation 
in LLN program) 
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Personal growth/ personal change (extent 
to which a learner perceives they have 
grown as a person as the result of 
engagement in a LLN program) 
















Social Capital (extent of changes in 
community participation, involvement in 
networks, etc, as a result of participation 
in LLN program) 
















Other suggestion: 
















20. To what degree do you think the 
areas which you have nominated as 
Strongly Agree and Agree result from the 
students’ participation in this course? 

None (1), Very little (2), Some (3), Quite a bit 
(4), Very much (5 ) ) Discuss responses 
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COORDINATORS/TEACHERS/TUTORS - EXAMPLES OF ASSESSSMENT TOOLS 



We would appreciate your views on the examples of the assessment tools which were attached to 
the Summary of Research we sent to the Centre. These aren’t intended to be necessarily what we 
want to end up with this in this project. They’re just a range of instruments we found in our 
research and your opinions will help decide whether any of them are worth considering. Please 
give reasons for your responses. 

21. There are five possible responses to the usefulness of each example: Not useful (1), Limited 
usefulness (2), Uncertain (3), Useful (4) andUery useful (5). 



Teachers’ ratings (1-5) 



Tool/ Example 


name 


name 


name 


name 


name 


name 


name 


name 


name 


Appendix 1: Outcomes for 
providers/ stakeholders/ funders 
& Outcomes experienced by 
the learner (Foster et al 1997) 




















Appendix 2: Matrix classifying 
the effects of learning (Schuller 
et al 2004) 




















Appendix 3: Getting to know 
you (SOUL) (Anderson et al 
2006) 




















Appendix 4: Indicators of 
success identified by students 
(Eldred 2002) 




















Appendix 5: Emergent criteria 
of achievement (Charnley and 
jones 1978) 




















Appendix 6: Catching 
confidence (Eldred et al 2004) 




















Appendix 7: Assessment 
profile* (Department for Work 
and Pensions 2004) 




















Appendix 8 — Personal 
development plan (Department 
for Work and Pensions 2004) 




















Appendix 9: Soft indicators: 
Individual profiling 
(Department for Work and 
Pensions 2004) 
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*This is an example of the wheel idea, based on rankings against indicators — the indicators would 
be different for L&N learning. 



22. In thinking of the course/ s for which you are responsible, do you think it might be possible 
to develop some indicators of progress of wider learning outcomes that are applicable across all 
courses or do you think you would need to adapt them to particular courses. 
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